From 16753a78f4e0f77d1542921a62b1758014d8e718 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Sun, 7 Nov 2021 08:07:57 -0800 Subject: [PATCH 01/14] Update navigation.yml Add GitHub link to top bar --- docs/_data/navigation.yml | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml index ad2e55fd..63058963 100644 --- a/docs/_data/navigation.yml +++ b/docs/_data/navigation.yml @@ -3,6 +3,8 @@ main: url: /news - title: Getting Started url: /getting-started + - title: GitHub + url: https://github.com/Roblox/luau pages: - title: Getting Started From f27580c74b09c9f0bdbc00aa77f5a3d79cc0faee Mon Sep 17 00:00:00 2001 From: ThePotato <10260415+ThePotato97@users.noreply.github.com> Date: Mon, 8 Nov 2021 07:19:28 +0000 Subject: [PATCH 02/14] Fix linux typo in README.md "Linus" to "Linux" (#174) --- README.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/README.md b/README.md index 020ac34b..ffb464d0 100644 --- a/README.md +++ b/README.md @@ -31,7 +31,7 @@ cmake --build . --target Luau.Repl.CLI --config RelWithDebInfo cmake --build . --target Luau.Analyze.CLI --config RelWithDebInfo ``` -Alternatively, on Linus/macOS you can use make: +Alternatively, on Linux/macOS you can use make: ```sh make config=release luau luau-analyze From 46244d2ea7223bbe0756b21e8b75b7fd5b8d6eb0 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Sun, 7 Nov 2021 23:21:34 -0800 Subject: [PATCH 03/14] Update getting-started.md Update file extension from .lua to .luau Contributes to #97 --- docs/_pages/getting-started.md | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/docs/_pages/getting-started.md b/docs/_pages/getting-started.md index 2f6017da..f64e0ebc 100644 --- a/docs/_pages/getting-started.md +++ b/docs/_pages/getting-started.md @@ -8,7 +8,7 @@ To get started with Luau you need to use `luau` command line binary to run your ## Creating a script -To create your own testing script, create a new file with `.lua` as the extension: +To create your own testing script, create a new file with `.luau` as the extension: ```lua function ispositive(x) @@ -26,7 +26,7 @@ print(isfoo("bar")) print(isfoo(1)) ``` -You can now run the file using `luau test.lua` and analyze it using `luau-analyze test.lua`. +You can now run the file using `luau test.luau` and analyze it using `luau-analyze test.luau`. Note that there are no warnings about calling ``ispositive()`` with a string, or calling ``isfoo()`` a number. This is because the type checking uses non-strict mode by default, which is lenient in how it infers types used by the program. @@ -52,8 +52,8 @@ In this case, Luau will use the ``return x > 0`` statement to infer that ``ispos Based on Luau's type inference, the analysis tool will now flag the incorrect call to ``ispositive()``: ``` -$ luau-analyze test.lua -test.lua(7,18): TypeError: Type 'string' could not be converted into 'number' +$ luau-analyze test.luau +test.luau(7,18): TypeError: Type 'string' could not be converted into 'number' ``` ## Annotations @@ -92,9 +92,9 @@ result = ispositive(1) Oops -- we're returning string values, but we forgot to update the function return type. Since we've told Luau that ``ispositive()`` returns a boolean (and that's how we're using it), the call site isn't flagged as an error. But because the annotation doesn't match our code, we get a warning in the function body itself: ``` -$ luau-analyze test.lua -test.lua(5,9): TypeError: Type 'string' could not be converted into 'boolean' -test.lua(7,9): TypeError: Type 'string' could not be converted into 'boolean' +$ luau-analyze test.luau +test.luau(5,9): TypeError: Type 'string' could not be converted into 'boolean' +test.luau(7,9): TypeError: Type 'string' could not be converted into 'boolean' ``` The fix is simple; just change the annotation to declare the return type as a string: @@ -117,8 +117,8 @@ result = ispositive(1) Well, almost - since we declared ``result`` as a boolean, the call site is now flagged: ``` -$ luau-analyze test.lua -test.lua(12,10): TypeError: Type 'string' could not be converted into 'boolean' +$ luau-analyze test.luau +test.luau(12,10): TypeError: Type 'string' could not be converted into 'boolean' ``` If we update the type of the local variable, everything is good. Note that we could also just let Luau infer the type of ``result`` by changing it to the single line version ``local result = ispositive(1)``. From f1649a43cdf7dd9e41496913672e82b30c067234 Mon Sep 17 00:00:00 2001 From: LoganDark Date: Mon, 8 Nov 2021 08:26:03 -0800 Subject: [PATCH 04/14] Add userdata to lua_Callbacks (#168) --- VM/include/lua.h | 2 ++ 1 file changed, 2 insertions(+) diff --git a/VM/include/lua.h b/VM/include/lua.h index 2f93ad90..8e7d6468 100644 --- a/VM/include/lua.h +++ b/VM/include/lua.h @@ -346,6 +346,8 @@ struct lua_Debug * can only be changed when the VM is not running any code */ struct lua_Callbacks { + void* userdata; /* arbitrary userdata pointer that is never overwritten by Luau */ + void (*interrupt)(lua_State* L, int gc); /* gets called at safepoints (loop back edges, call/ret, gc) if set */ void (*panic)(lua_State* L, int errcode); /* gets called when an unprotected error is raised (if longjmp is used) */ From 7ca09ec100b7583eaf26adb6ceb5fbff42f50db0 Mon Sep 17 00:00:00 2001 From: LoganDark Date: Mon, 8 Nov 2021 11:09:30 -0800 Subject: [PATCH 05/14] Fix lbuiltins.cpp comment (#180) --- VM/src/lbuiltins.cpp | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/VM/src/lbuiltins.cpp b/VM/src/lbuiltins.cpp index e1c99b21..9ff0d6a8 100644 --- a/VM/src/lbuiltins.cpp +++ b/VM/src/lbuiltins.cpp @@ -20,8 +20,9 @@ // If types of the arguments mismatch, luauF_* needs to return -1 and the execution will fall back to the usual call path // If luauF_* succeeds, it needs to return *all* requested arguments, filling results with nil as appropriate. // On input, nparams refers to the actual number of arguments (0+), whereas nresults contains LUA_MULTRET for arbitrary returns or 0+ for a -// fixed-length return Because of this, and the fact that "extra" returned values will be ignored, implementations below typically check that nresults -// is <= expected number, which covers the LUA_MULTRET case. +// fixed-length return +// Because of this, and the fact that "extra" returned values will be ignored, implementations below typically check that nresults is <= expected +// number, which covers the LUA_MULTRET case. static int luauF_assert(lua_State* L, StkId res, TValue* arg0, int nresults, StkId args, int nparams) { From 7816f2596f0545629c3c303ca1d7aee46025c693 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Mon, 8 Nov 2021 15:01:57 -0800 Subject: [PATCH 06/14] Update config.yml Rename "help and support" to "questions" --- .github/ISSUE_TEMPLATE/config.yml | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/.github/ISSUE_TEMPLATE/config.yml b/.github/ISSUE_TEMPLATE/config.yml index 37382f13..55f9db39 100644 --- a/.github/ISSUE_TEMPLATE/config.yml +++ b/.github/ISSUE_TEMPLATE/config.yml @@ -1,5 +1,5 @@ blank_issues_enabled: false contact_links: - - name: Help and support + - name: Questions url: https://github.com/Roblox/luau/discussions about: Please use GitHub Discussions if you have questions or need help. From 773fb5bd6f9ef512d61b87fb13cd88ffc0af3581 Mon Sep 17 00:00:00 2001 From: LoganDark Date: Mon, 8 Nov 2021 15:30:51 -0800 Subject: [PATCH 07/14] Don't allow registry to be set to readonly (#184) --- VM/src/lapi.cpp | 1 + 1 file changed, 1 insertion(+) diff --git a/VM/src/lapi.cpp b/VM/src/lapi.cpp index f2e97c66..c0c78b93 100644 --- a/VM/src/lapi.cpp +++ b/VM/src/lapi.cpp @@ -703,6 +703,7 @@ void lua_setreadonly(lua_State* L, int objindex, bool value) const TValue* o = index2adr(L, objindex); api_check(L, ttistable(o)); Table* t = hvalue(o); + api_check(L, t != hvalue(registry(L))); t->readonly = value; return; } From 9523670e631751b70ed3dc07641a0d7fc9989ce1 Mon Sep 17 00:00:00 2001 From: LoganDark Date: Tue, 9 Nov 2021 07:06:25 -0800 Subject: [PATCH 08/14] Remove Roblox-specific mutable globals (#185) Instead the code that calls the compiler needs to use Compiler::mutableGlobals to disable GETIMPORT optimization. --- Compiler/include/Luau/Compiler.h | 4 ++ Compiler/src/Compiler.cpp | 30 +++++--- tests/Compiler.test.cpp | 115 +++++++++++++++++++++++++++++++ 3 files changed, 141 insertions(+), 8 deletions(-) diff --git a/Compiler/include/Luau/Compiler.h b/Compiler/include/Luau/Compiler.h index f8d67158..2935f595 100644 --- a/Compiler/include/Luau/Compiler.h +++ b/Compiler/include/Luau/Compiler.h @@ -36,6 +36,10 @@ struct CompileOptions // global builtin to construct vectors; disabled by default const char* vectorLib = nullptr; const char* vectorCtor = nullptr; + + // array of globals that are mutable; disables the import optimization for fields accessed through them + // use NULL to end the array + const char** mutableGlobals = nullptr; }; class CompileError : public std::exception diff --git a/Compiler/src/Compiler.cpp b/Compiler/src/Compiler.cpp index 45d10cc3..08cb9934 100644 --- a/Compiler/src/Compiler.cpp +++ b/Compiler/src/Compiler.cpp @@ -13,6 +13,7 @@ LUAU_FASTFLAGVARIABLE(LuauPreloadClosures, false) LUAU_FASTFLAGVARIABLE(LuauPreloadClosuresFenv, false) LUAU_FASTFLAGVARIABLE(LuauPreloadClosuresUpval, false) +LUAU_FASTFLAGVARIABLE(LuauGenericSpecialGlobals, false) LUAU_FASTFLAG(LuauIfElseExpressionBaseSupport) namespace Luau @@ -1277,7 +1278,7 @@ struct Compiler { const Global* global = globals.find(expr->name); - return options.optimizationLevel >= 1 && (!global || (!global->written && !global->special)); + return options.optimizationLevel >= 1 && (!global || (!global->written && !global->writable)); } void compileExprIndexName(AstExprIndexName* expr, uint8_t target) @@ -3447,7 +3448,7 @@ struct Compiler struct Global { - bool special = false; + bool writable = false; bool written = false; }; @@ -3505,7 +3506,7 @@ struct Compiler { Global* g = globals.find(object->name); - return !g || (!g->special && !g->written) ? Builtin{object->name, expr->index} : Builtin(); + return !g || (!g->writable && !g->written) ? Builtin{object->name, expr->index} : Builtin(); } else { @@ -3703,13 +3704,26 @@ void compileOrThrow(BytecodeBuilder& bytecode, AstStatBlock* root, const AstName Compiler compiler(bytecode, options); - // since access to some global objects may result in values that change over time, we block table imports - for (const char* global : kSpecialGlobals) + // since access to some global objects may result in values that change over time, we block imports from non-readonly tables + if (FFlag::LuauGenericSpecialGlobals) { - AstName name = names.get(global); + if (AstName name = names.get("_G"); name.value) + compiler.globals[name].writable = true; - if (name.value) - compiler.globals[name].special = true; + if (options.mutableGlobals) + for (const char** ptr = options.mutableGlobals; *ptr != NULL; ++ptr) + { + if (AstName name = names.get(*ptr); name.value) + compiler.globals[name].writable = true; + } + } + else + { + for (const char* global : kSpecialGlobals) + { + if (AstName name = names.get(global); name.value) + compiler.globals[name].writable = true; + } } // this visitor traverses the AST to analyze mutability of locals/globals, filling Local::written and Global::written diff --git a/tests/Compiler.test.cpp b/tests/Compiler.test.cpp index 1389bfec..7f03019c 100644 --- a/tests/Compiler.test.cpp +++ b/tests/Compiler.test.cpp @@ -13,6 +13,7 @@ LUAU_FASTFLAG(LuauPreloadClosures) LUAU_FASTFLAG(LuauPreloadClosuresFenv) LUAU_FASTFLAG(LuauPreloadClosuresUpval) +LUAU_FASTFLAG(LuauGenericSpecialGlobals) using namespace Luau; @@ -3670,4 +3671,118 @@ RETURN R0 0 )"); } +TEST_CASE("LuauGenericSpecialGlobals") +{ + const char* source = R"( +print() +Game.print() +Workspace.print() +_G.print() +game.print() +plugin.print() +script.print() +shared.print() +workspace.print() +)"; + + { + ScopedFastFlag genericSpecialGlobals{"LuauGenericSpecialGlobals", false}; + + // Check Roblox globals are here + CHECK_EQ("\n" + compileFunction0(source), R"( +GETIMPORT R0 1 +CALL R0 0 0 +GETIMPORT R1 3 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 5 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 7 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 9 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 11 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 13 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 15 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 17 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +RETURN R0 0 +)"); + } + + ScopedFastFlag genericSpecialGlobals{"LuauGenericSpecialGlobals", true}; + + // Check Roblox globals are no longer here + CHECK_EQ("\n" + compileFunction0(source), R"( +GETIMPORT R0 1 +CALL R0 0 0 +GETIMPORT R0 3 +CALL R0 0 0 +GETIMPORT R0 5 +CALL R0 0 0 +GETIMPORT R1 7 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R0 9 +CALL R0 0 0 +GETIMPORT R0 11 +CALL R0 0 0 +GETIMPORT R0 13 +CALL R0 0 0 +GETIMPORT R0 15 +CALL R0 0 0 +GETIMPORT R0 17 +CALL R0 0 0 +RETURN R0 0 +)"); + + // Check we can add them back + Luau::BytecodeBuilder bcb; + bcb.setDumpFlags(Luau::BytecodeBuilder::Dump_Code); + Luau::CompileOptions options; + const char* mutableGlobals[] = {"Game", "Workspace", "game", "plugin", "script", "shared", "workspace", NULL}; + options.mutableGlobals = &mutableGlobals[0]; + Luau::compileOrThrow(bcb, source, options); + + CHECK_EQ("\n" + bcb.dumpFunction(0), R"( +GETIMPORT R0 1 +CALL R0 0 0 +GETIMPORT R1 3 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 5 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 7 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 9 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 11 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 13 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 15 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +GETIMPORT R1 17 +GETTABLEKS R0 R1 K0 +CALL R0 0 0 +RETURN R0 0 +)"); +} + TEST_SUITE_END(); From fe1bf43b546784b257f50380c836d12b06004793 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 9 Nov 2021 10:40:14 -0800 Subject: [PATCH 09/14] RFC: bit32.countlz/countrz (#89) Co-authored-by: vegorov-rbx <75688451+vegorov-rbx@users.noreply.github.com> --- rfcs/function-bit32-countlz-countrz.md | 50 ++++++++++++++++++++++++++ 1 file changed, 50 insertions(+) create mode 100644 rfcs/function-bit32-countlz-countrz.md diff --git a/rfcs/function-bit32-countlz-countrz.md b/rfcs/function-bit32-countlz-countrz.md new file mode 100644 index 00000000..986da70c --- /dev/null +++ b/rfcs/function-bit32-countlz-countrz.md @@ -0,0 +1,50 @@ +# bit32.countlz/countrz + +## Summary + +Add bit32.countlz (count left zeroes) and bit32.countrz (count right zeroes) to accelerate bit scanning + +## Motivation + +All CPUs have instructions to determine the position of first/last set bit in an integer. These instructions have a variety of uses, the popular ones being: + +- Fast implementation of integer logarithm (essentially allowing to compute `floor(log2(value))` quickly) +- Scanning set bits in an integer, which allows efficient traversal of compact representation of bitmaps +- Allocating bits out of a bitmap quickly + +Today it's possible to approximate `countlz` using `floor` and `log` but this approximation is relatively slow; approximating `countrz` is difficult without iterating through each bit. + +## Design + +`bit32` library will gain two new functions, `countlz` and `countrz`: + +``` +function bit32.countlz(n: number): number +function bit32.countrz(n: number): number +``` + +`countlz` takes an integer number (converting the input number to a 32-bit unsigned integer as all other `bit32` functions do), and returns the number of consecutive left-most zero bits - that is, the number of most significant zero bits in a 32-bit number until the first 1. The result is in `[0, 32]` range. + +For example, when the input number is `0`, it's `32`. When the input number is `2^k`, the result is `31-k`. + +`countrz` takes an integer number (converting the input number to a 32-bit unsigned integer as all other `bit32` functions do), and returns the number of consecutive right-most zero bits - that is, +the number of least significant zero bits in a 32-bit number until the first 1. The result is in `[0, 32]` range. + +For example, when the input number is `0`, it's `32`. When the input number is `2^k`, the result is `k`. + +> Non-normative: a proof of concept implementation shows that a polyfill for `countlz` takes ~34 ns per loop iteration when computing `countlz` for an increasing number sequence, whereas +> a builtin implementation takes ~4 ns. + +## Drawbacks + +None known. + +## Alternatives + +These functions can be alternatively specified as "find the position of the most/least significant bit set" (e.g. "ffs"/"fls" for "find first set"/"find last set"). This formulation +can be more immediately useful since the bit position is usually more important than the number of bits. However, the bit position is undefined when the input number is zero, +returning a sentinel such as -1 seems non-idiomatic, and returning `nil` seems awkward for calling code. Counting functions don't have this problem. + +An early version of this proposal suggested `clz`/`ctz` (leading/trailing) as names; however, using a full verb is more consistent with other operations like shift/rotate, and left/right may be easier to understand intuitively compared to leading/trailing. left/right are used by C++20. + +Of the two functions, `countlz` is vastly more useful than `countrz`; we could implement just `countlz`, but having both is nice for symmetry. From 3ba0bdf7084f3d8eb37c64f0872d4ed912edede8 Mon Sep 17 00:00:00 2001 From: NotDSF <67711164+NotDSF@users.noreply.github.com> Date: Tue, 9 Nov 2021 22:54:53 +0000 Subject: [PATCH 10/14] Save bytecode to file through CLI (#170) Using --compile=binary it's now possible to produce binary bytecode so that it can be compiled offline and loaded into the VM. --- CLI/Repl.cpp | 46 +++++++++++++++++++++++++++++++++++++++------- 1 file changed, 39 insertions(+), 7 deletions(-) diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 323ab144..a5bd5737 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -13,6 +13,17 @@ #include +#ifdef _WIN32 + #include + #include +#endif + +enum class CompileFormat +{ + Default, + Binary +}; + static int lua_loadstring(lua_State* L) { size_t l = 0; @@ -370,7 +381,7 @@ static void reportError(const char* name, const Luau::CompileError& error) report(name, error.getLocation(), "CompileError", error.what()); } -static bool compileFile(const char* name) +static bool compileFile(const char* name, CompileFormat format) { std::optional source = readFile(name); if (!source) @@ -387,7 +398,20 @@ static bool compileFile(const char* name) Luau::compileOrThrow(bcb, *source); - printf("%s", bcb.dumpEverything().c_str()); + switch (format) + { + case CompileFormat::Default: + printf("%s", bcb.dumpEverything().c_str()); + break; + case CompileFormat::Binary: + #ifdef _WIN32 + _setmode(_fileno(stdout), _O_BINARY); + #endif + + std::string Bytecode = bcb.getBytecode(); + fwrite(Bytecode.c_str(), 1, Bytecode.size(), stdout); + break; + } return true; } @@ -412,7 +436,7 @@ static void displayHelp(const char* argv0) printf("\n"); printf("Available modes:\n"); printf(" omitted: compile and run input files one by one\n"); - printf(" --compile: compile input files and output resulting bytecode\n"); + printf(" --compile[=format]: compile input files and output resulting formatted bytecode (binary or text)\n"); printf("\n"); printf("Available options:\n"); printf(" --profile[=N]: profile the code using N Hz sampling (default 10000) and output results to profile.out\n"); @@ -444,8 +468,16 @@ int main(int argc, char** argv) return 0; } - if (argc >= 2 && strcmp(argv[1], "--compile") == 0) - { + + if (argc >= 2 && strncmp(argv[1], "--compile", strlen("--compile")) == 0) + { + CompileFormat format = CompileFormat::Default; + + if (strcmp(argv[1], "--compile=binary") == 0) + { + format = CompileFormat::Binary; + } + int failed = 0; for (int i = 2; i < argc; ++i) @@ -457,12 +489,12 @@ int main(int argc, char** argv) { traverseDirectory(argv[i], [&](const std::string& name) { if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) - failed += !compileFile(name.c_str()); + failed += !compileFile(name.c_str(), format); }); } else { - failed += !compileFile(argv[i]); + failed += !compileFile(argv[i], format); } } From f3468be92b90f8d4b3069a0bc6e8b30d04a686cd Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 9 Nov 2021 15:11:52 -0800 Subject: [PATCH 11/14] Small follow code cleanup for Repl.cpp --- CLI/Repl.cpp | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-) diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index a5bd5737..1c5305a4 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -14,8 +14,8 @@ #include #ifdef _WIN32 - #include - #include +#include +#include #endif enum class CompileFormat @@ -404,12 +404,7 @@ static bool compileFile(const char* name, CompileFormat format) printf("%s", bcb.dumpEverything().c_str()); break; case CompileFormat::Binary: - #ifdef _WIN32 - _setmode(_fileno(stdout), _O_BINARY); - #endif - - std::string Bytecode = bcb.getBytecode(); - fwrite(Bytecode.c_str(), 1, Bytecode.size(), stdout); + fwrite(bcb.getBytecode().data(), 1, bcb.getBytecode().size(), stdout); break; } @@ -470,13 +465,16 @@ int main(int argc, char** argv) if (argc >= 2 && strncmp(argv[1], "--compile", strlen("--compile")) == 0) - { + { CompileFormat format = CompileFormat::Default; - if (strcmp(argv[1], "--compile=binary") == 0) - { + if (strcmp(argv[1], "--compile=binary") == 0) format = CompileFormat::Binary; - } + +#ifdef _WIN32 + if (format == CompileFormat::Binary) + _setmode(_fileno(stdout), _O_BINARY); +#endif int failed = 0; @@ -488,7 +486,9 @@ int main(int argc, char** argv) if (isDirectory(argv[i])) { traverseDirectory(argv[i], [&](const std::string& name) { - if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) + if (name.length() > 5 && name.rfind(".luau") == name.length() - 5) + failed += !compileFile(name.c_str(), format); + else if (name.length() > 4 && name.rfind(".lua") == name.length() - 4) failed += !compileFile(name.c_str(), format); }); } From d6b3346f580b4b50fbdcdfe0ff748f65204c8214 Mon Sep 17 00:00:00 2001 From: LoganDark Date: Tue, 9 Nov 2021 17:02:46 -0800 Subject: [PATCH 12/14] move static_assert from ltable.h to ltable.cpp (#189) --- VM/src/ltable.cpp | 1 + VM/src/ltable.h | 1 - 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/VM/src/ltable.cpp b/VM/src/ltable.cpp index 883442ae..07d22d59 100644 --- a/VM/src/ltable.cpp +++ b/VM/src/ltable.cpp @@ -30,6 +30,7 @@ LUAU_FASTFLAGVARIABLE(LuauArrayBoundary, false) #define MAXBITS 26 #define MAXSIZE (1 << MAXBITS) +static_assert(offsetof(LuaNode, val) == 0, "Unexpected Node memory layout, pointer cast in gval2slot is incorrect"); // TKey is bitpacked for memory efficiency so we need to validate bit counts for worst case static_assert(TKey{{NULL}, 0, LUA_TDEADKEY, 0}.tt == LUA_TDEADKEY, "not enough bits for tt"); static_assert(TKey{{NULL}, 0, LUA_TNIL, MAXSIZE - 1}.next == MAXSIZE - 1, "not enough bits for next"); diff --git a/VM/src/ltable.h b/VM/src/ltable.h index f98d87b1..45061443 100644 --- a/VM/src/ltable.h +++ b/VM/src/ltable.h @@ -9,7 +9,6 @@ #define gval(n) (&(n)->val) #define gnext(n) ((n)->key.next) -static_assert(offsetof(LuaNode, val) == 0, "Unexpected Node memory layout, pointer cast below is incorrect"); #define gval2slot(t, v) int(cast_to(LuaNode*, static_cast(v)) - t->node) LUAI_FUNC const TValue* luaH_getnum(Table* t, int key); From aec8fbfd0facdba1e1f6dcb070742000d92fa5ed Mon Sep 17 00:00:00 2001 From: Pelanyo Kamara Date: Wed, 10 Nov 2021 16:40:46 +0000 Subject: [PATCH 13/14] Feature: Web REPL using Emscripten (#138) Currently doesn't include the new page into navigation since we aren't building the .js files anywhere. --- .gitignore | 1 + CLI/Repl.cpp | 44 ++++++++++++++++++++++++++++++++++ CMakeLists.txt | 38 +++++++++++++++++++++-------- docs/_data/navigation.yml | 4 ++++ docs/_includes/repl.html | 50 +++++++++++++++++++++++++++++++++++++++ docs/_pages/demo.md | 6 +++++ 6 files changed, 133 insertions(+), 10 deletions(-) create mode 100644 docs/_includes/repl.html create mode 100644 docs/_pages/demo.md diff --git a/.gitignore b/.gitignore index 0b2422ce..fa11b45b 100644 --- a/.gitignore +++ b/.gitignore @@ -5,3 +5,4 @@ ^default.prof* ^fuzz-* ^luau$ +/.vs diff --git a/CLI/Repl.cpp b/CLI/Repl.cpp index 1c5305a4..63b86a80 100644 --- a/CLI/Repl.cpp +++ b/CLI/Repl.cpp @@ -198,6 +198,11 @@ static std::string runCode(lua_State* L, const std::string& source) error += "\nstack backtrace:\n"; error += lua_debugtrace(T); +#ifdef __EMSCRIPTEN__ + // nicer formatting for errors in web repl + error = "Error:" + error; +#endif + fprintf(stdout, "%s", error.c_str()); } @@ -205,6 +210,44 @@ static std::string runCode(lua_State* L, const std::string& source) return std::string(); } +#ifdef __EMSCRIPTEN__ +extern "C" +{ + const char* executeScript(const char* source) + { + // static string for caching result (prevents dangling ptr on function exit) + static std::string result; + + // setup flags + for (Luau::FValue* flag = Luau::FValue::list; flag; flag = flag->next) + if (strncmp(flag->name, "Luau", 4) == 0) + flag->value = true; + + // create new state + std::unique_ptr globalState(luaL_newstate(), lua_close); + lua_State* L = globalState.get(); + + // setup state + setupState(L); + + // sandbox thread + luaL_sandboxthread(L); + + // run code + collect error + std::string error = runCode(L, source); + result = error; + + if (error.length()) + { + return result.c_str(); + } + return NULL; + } +} +#endif + +// Excluded from emscripten compilation to avoid -Wunused-function errors. +#ifndef __EMSCRIPTEN__ static void completeIndexer(lua_State* L, const char* editBuffer, size_t start, std::vector& completions) { std::string_view lookup = editBuffer + start; @@ -547,3 +590,4 @@ int main(int argc, char** argv) return failed; } } +#endif diff --git a/CMakeLists.txt b/CMakeLists.txt index d6598f2a..36014a98 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -17,17 +17,26 @@ add_library(Luau.VM STATIC) if(LUAU_BUILD_CLI) add_executable(Luau.Repl.CLI) - add_executable(Luau.Analyze.CLI) + if(NOT EMSCRIPTEN) + add_executable(Luau.Analyze.CLI) + else() + # add -fexceptions for emscripten to allow exceptions to be caught in C++ + set(CMAKE_CXX_FLAGS "${CMAKE_CXX_FLAGS} -fexceptions") + endif() # This also adds target `name` on Linux/macOS and `name.exe` on Windows set_target_properties(Luau.Repl.CLI PROPERTIES OUTPUT_NAME luau) - set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) + + if(NOT EMSCRIPTEN) + set_target_properties(Luau.Analyze.CLI PROPERTIES OUTPUT_NAME luau-analyze) + endif() endif() -if(LUAU_BUILD_TESTS) +if(LUAU_BUILD_TESTS AND NOT EMSCRIPTEN) add_executable(Luau.UnitTest) add_executable(Luau.Conformance) endif() + include(Sources.cmake) target_compile_features(Luau.Ast PUBLIC cxx_std_17) @@ -53,10 +62,6 @@ if(MSVC) else() list(APPEND LUAU_OPTIONS -Wall) # All warnings list(APPEND LUAU_OPTIONS -Werror) # Warnings are errors - - if(CMAKE_CXX_COMPILER_ID STREQUAL "GNU") - list(APPEND LUAU_OPTIONS -Wno-unused) # GCC considers variables declared/checked in if() as unused - endif() endif() target_compile_options(Luau.Ast PRIVATE ${LUAU_OPTIONS}) @@ -65,7 +70,10 @@ target_compile_options(Luau.VM PRIVATE ${LUAU_OPTIONS}) if(LUAU_BUILD_CLI) target_compile_options(Luau.Repl.CLI PRIVATE ${LUAU_OPTIONS}) - target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) + + if(NOT EMSCRIPTEN) + target_compile_options(Luau.Analyze.CLI PRIVATE ${LUAU_OPTIONS}) + endif() target_include_directories(Luau.Repl.CLI PRIVATE extern) target_link_libraries(Luau.Repl.CLI PRIVATE Luau.Compiler Luau.VM) @@ -74,10 +82,20 @@ if(LUAU_BUILD_CLI) target_link_libraries(Luau.Repl.CLI PRIVATE pthread) endif() - target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) + if(NOT EMSCRIPTEN) + target_link_libraries(Luau.Analyze.CLI PRIVATE Luau.Analysis) + endif() + + if(EMSCRIPTEN) + # declare exported functions to emscripten + target_link_options(Luau.Repl.CLI PRIVATE -sEXPORTED_FUNCTIONS=['_executeScript'] -sEXPORTED_RUNTIME_METHODS=['ccall','cwrap'] -fexceptions) + + # custom output directory for wasm + js file + set_target_properties(Luau.Repl.CLI PROPERTIES RUNTIME_OUTPUT_DIRECTORY ${CMAKE_SOURCE_DIR}/docs/assets/luau) + endif() endif() -if(LUAU_BUILD_TESTS) +if(LUAU_BUILD_TESTS AND NOT EMSCRIPTEN) target_compile_options(Luau.UnitTest PRIVATE ${LUAU_OPTIONS}) target_include_directories(Luau.UnitTest PRIVATE extern) target_link_libraries(Luau.UnitTest PRIVATE Luau.Analysis Luau.Compiler) diff --git a/docs/_data/navigation.yml b/docs/_data/navigation.yml index 63058963..c828b2d4 100644 --- a/docs/_data/navigation.yml +++ b/docs/_data/navigation.yml @@ -27,3 +27,7 @@ pages: url: /profile - title: Library url: /library + +# Remove demo pages until solution is found +# - title: Demo +# url: /demo diff --git a/docs/_includes/repl.html b/docs/_includes/repl.html new file mode 100644 index 00000000..73c5ba40 --- /dev/null +++ b/docs/_includes/repl.html @@ -0,0 +1,50 @@ +
+
+ +
+ +

+ + +
+

+
+ +
+ +

+ +
+
+ + + diff --git a/docs/_pages/demo.md b/docs/_pages/demo.md new file mode 100644 index 00000000..fefa4b20 --- /dev/null +++ b/docs/_pages/demo.md @@ -0,0 +1,6 @@ +--- +permalink: /demo +title: Demo +--- + +{% include repl.html %} From 4957812b62e22a589542eaa2bcc2938222cca89b Mon Sep 17 00:00:00 2001 From: dcope-rbx <91100513+dcope-rbx@users.noreply.github.com> Date: Wed, 10 Nov 2021 09:26:20 -0800 Subject: [PATCH 14/14] Added documentation related to typecasting (#191) --- docs/_pages/typecheck.md | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/docs/_pages/typecheck.md b/docs/_pages/typecheck.md index 6c00b548..fcc14e02 100644 --- a/docs/_pages/typecheck.md +++ b/docs/_pages/typecheck.md @@ -351,6 +351,30 @@ local onlyString: string = stringOrNumber -- ok local onlyNumber: number = stringOrNumber -- not ok ``` +## Typecasts + +Expressions may be typecast using `::`. Typecasting is useful for specifying the type of an expression when the automatically inferred type is too generic. + +For example, consider the following table constructor where the intent is to store a table of names: +```lua +local myTable = {names = {}} +table.insert(myTable.names, 42) -- Inserting a number ought to cause a type error, but doesn't +``` + +In order to specify the type of the `names` table a typecast may be used: + +```lua +local myTable = {names = {} :: {string}} +table.insert(myTable.names, 42) -- not ok, invalid 'number' to 'string' conversion +``` + +A typecast itself is also type checked to ensure the conversion specified is a valid one: +```lua +local number = 1 +local value = number :: any -- ok, 'number' is a subtype of 'any' +local flag = number :: boolean -- not ok, invalid 'number' to 'boolean' conversion +``` + ## Roblox types Roblox supports a rich set of classes and data types, [documented here](https://developer.roblox.com/en-us/api-reference). All of them are readily available for the type checker to use by their name (e.g. `Part` or `RaycastResult`). @@ -397,3 +421,10 @@ return module ``` There are some caveats here though. For instance, the require path must be resolvable statically, otherwise Luau cannot accurately type check it. + +### Cyclic module dependencies + +Cyclic module dependencies can cause problems for the type checker. In order to break a module dependency cycle a typecast of the module to `any` may be used: +```lua +local myModule = require(MyModule) :: any +``` \ No newline at end of file