luau/CodeGen/include/Luau/CodeGen.h

148 lines
5.4 KiB
C
Raw Normal View History

// This file is part of the Luau programming language and is licensed under MIT License; see LICENSE.txt for details
#pragma once
#include "Luau/CodeGenCommon.h"
#include "Luau/CodeGenOptions.h"
#include "Luau/LoweringStats.h"
#include <array>
#include <memory>
#include <string>
Sync to upstream/release/605 (#1118) - Implemented [Require by String with Relative Paths](https://github.com/luau-lang/rfcs/blob/master/docs/new-require-by-string-semantics.md) RFC - Implemented [Require by String with Aliases](https://github.com/luau-lang/rfcs/blob/master/docs/require-by-string-aliases.md) RFC with support for `paths` and `alias` arrays in .luarc - Added SUBRK and DIVRK bytecode instructions to speed up constant-number and constant/number operations - Added `--vector-lib`, `--vector-ctor` and `--vector-type` options to luau-compile to support code with vectors New Solver - Correctness fixes to subtyping - Improvements to dataflow analysis Native Code Generation - Added bytecode analysis pass to predict type tags used in operations - Fixed rare cases of numerical loops being generated without an interrupt instruction - Restored optimization data propagation into the linear block - Duplicate buffer length checks are optimized away Miscellaneous - Small performance improvements to new non-strict mode - Introduced more scripts for fuzzing Luau and processing the results, including fuzzer build support for CMake Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: David Cope <dcope@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vighnesh Vijay <vvijay@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com> --------- Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: David Cope <dcope@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2023-12-02 07:46:57 +00:00
#include <vector>
#include <stddef.h>
Sync to upstream/release/576 (#928) * `ClassType` can now have an indexer defined on it. This allows custom types to be used in `t[x]` expressions. * Fixed search for closest executable breakpoint line. Previously, breakpoints might have been skipped in `else` blocks at the end of a function * Fixed how unification is performed for two optional types `a? <: b?`, previously it might have unified either 'a' or 'b' with 'nil'. Note that this fix is not enabled by default yet (see the list in `ExperimentalFlags.h`) In the new type solver, a concept of 'Type Families' has been introduced. Type families can be thought of as type aliases with custom type inference/reduction logic included with them. For example, we can have an `Add<T, U>` type family that will resolve the type that is the result of adding two values together. This will help type inference to figure out what 'T' and 'U' might be when explicit type annotations are not provided. In this update we don't define any type families, but they will be added in the near future. It is also possible for Luau embedders to define their own type families in the global/environment scope. Other changes include: * Fixed scope used to find out which generic types should be included in the function generic type list * Fixed a crash after cyclic bound types were created during unification And in native code generation (jit): * Use of arm64 target on M1 now requires macOS 13 * Entry into native code has been optimized. This is especially important for coroutine call/pcall performance as they involve going through a C call frame * LOP_LOADK(X) translation into IR has been improved to enable type tag/constant propagation * arm64 can use integer immediate values to synthesize floating-point values * x64 assembler removes duplicate 64bit numbers from the data section to save space * Linux `perf` can now be used to profile native Luau code (when running with --codegen-perf CLI argument)
2023-05-12 18:50:47 +01:00
#include <stdint.h>
struct lua_State;
namespace Luau
{
namespace CodeGen
{
// These enum values can be reported through telemetry.
// To ensure consistency, changes should be additive.
enum class CodeGenCompilationResult
{
Success = 0, // Successfully generated code for at least one function
NothingToCompile = 1, // There were no new functions to compile
NotNativeModule = 2, // Module does not have `--!native` comment
CodeGenNotInitialized = 3, // Native codegen system is not initialized
CodeGenOverflowInstructionLimit = 4, // Instruction limit overflow
CodeGenOverflowBlockLimit = 5, // Block limit overflow
CodeGenOverflowBlockInstructionLimit = 6, // Block instruction limit overflow
CodeGenAssemblerFinalizationFailure = 7, // Failure during assembler finalization
CodeGenLoweringFailure = 8, // Lowering failed
AllocationFailed = 9, // Native codegen failed due to an allocation error
Count = 10,
};
std::string toString(const CodeGenCompilationResult& result);
struct ProtoCompilationFailure
{
CodeGenCompilationResult result = CodeGenCompilationResult::Success;
std::string debugname;
int line = -1;
};
struct CompilationResult
{
CodeGenCompilationResult result = CodeGenCompilationResult::Success;
std::vector<ProtoCompilationFailure> protoFailures;
[[nodiscard]] bool hasErrors() const
{
return result != CodeGenCompilationResult::Success || !protoFailures.empty();
}
};
struct CompilationStats
{
size_t bytecodeSizeBytes = 0;
size_t nativeCodeSizeBytes = 0;
size_t nativeDataSizeBytes = 0;
size_t nativeMetadataSizeBytes = 0;
uint32_t functionsTotal = 0;
uint32_t functionsCompiled = 0;
uint32_t functionsBound = 0;
};
bool isSupported();
class SharedCodeGenContext;
struct SharedCodeGenContextDeleter
{
void operator()(const SharedCodeGenContext* context) const noexcept;
};
using UniqueSharedCodeGenContext = std::unique_ptr<SharedCodeGenContext, SharedCodeGenContextDeleter>;
// Creates a new SharedCodeGenContext that can be used by multiple Luau VMs
// concurrently, using either the default allocator parameters or custom
// allocator parameters.
[[nodiscard]] UniqueSharedCodeGenContext createSharedCodeGenContext();
[[nodiscard]] UniqueSharedCodeGenContext createSharedCodeGenContext(AllocationCallback* allocationCallback, void* allocationCallbackContext);
[[nodiscard]] UniqueSharedCodeGenContext createSharedCodeGenContext(
size_t blockSize,
size_t maxTotalSize,
AllocationCallback* allocationCallback,
void* allocationCallbackContext
);
// Destroys the provided SharedCodeGenContext. All Luau VMs using the
// SharedCodeGenContext must be destroyed before this function is called.
void destroySharedCodeGenContext(const SharedCodeGenContext* codeGenContext) noexcept;
Sync to upstream/release/627 (#1266) ### What's new? * Removed new `table.move` optimization because of correctness problems. ### New Type Solver * Improved error messages for type families to describe what's wrong in more detail, and ideally without using the term `type family` at all. * Change `boolean` and `string` singletons in type checking to report errors to the user when they've gotten an impossible type (indicating a type error from their context). * Split debugging flags for type family reduction (`DebugLuauLogTypeFamilies`) from general solver logging (`DebugLuauLogSolver`). * Improve type simplification to support patterns like `(number | string) | (string | number)` becoming `number | string`. ### Native Code Generation * Use templated `luaV_doarith` to speedup vector operation fallbacks. * Various small changes to better support arm64 on Windows. ### Internal Contributors Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: James McNellis <jmcnellis@roblox.com> Co-authored-by: Vighnesh Vijay <vvijay@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com> --------- Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Vighnesh <vvijay@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: David Cope <dcope@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-05-26 18:09:09 +01:00
// Initializes native code-gen on the provided Luau VM, using a VM-specific
// code-gen context and either the default allocator parameters or custom
// allocator parameters.
void create(lua_State* L);
Sync to upstream/release/627 (#1266) ### What's new? * Removed new `table.move` optimization because of correctness problems. ### New Type Solver * Improved error messages for type families to describe what's wrong in more detail, and ideally without using the term `type family` at all. * Change `boolean` and `string` singletons in type checking to report errors to the user when they've gotten an impossible type (indicating a type error from their context). * Split debugging flags for type family reduction (`DebugLuauLogTypeFamilies`) from general solver logging (`DebugLuauLogSolver`). * Improve type simplification to support patterns like `(number | string) | (string | number)` becoming `number | string`. ### Native Code Generation * Use templated `luaV_doarith` to speedup vector operation fallbacks. * Various small changes to better support arm64 on Windows. ### Internal Contributors Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: James McNellis <jmcnellis@roblox.com> Co-authored-by: Vighnesh Vijay <vvijay@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com> --------- Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Vighnesh <vvijay@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: David Cope <dcope@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-05-26 18:09:09 +01:00
void create(lua_State* L, AllocationCallback* allocationCallback, void* allocationCallbackContext);
void create(lua_State* L, size_t blockSize, size_t maxTotalSize, AllocationCallback* allocationCallback, void* allocationCallbackContext);
// Initializes native code-gen on the provided Luau VM, using the provided
// SharedCodeGenContext. Note that after this function is called, the
// SharedCodeGenContext must not be destroyed until after the Luau VM L is
// destroyed via lua_close.
void create(lua_State* L, SharedCodeGenContext* codeGenContext);
Sync to upstream/release/617 (#1204) # What's Changed * Fix a case where the stack wasn't completely cleaned up where `debug.info` errored when passed `"f"` option and a thread. * Fix a case of uninitialized field in `luaF_newproto`. ### New Type Solver * When a local is captured in a function, don't add a new entry to the `DfgScope::bindings` if the capture occurs within a loop. * Fix a poor performance characteristic during unification by not trying to simplify an intersection. * Fix a case of multiple constraints mutating the same blocked type causing incorrect inferences. * Fix a case of assertion failure when overload resolution encounters a return typepack mismatch. * When refining a property of the top `table` type, we no longer signal an unknown property error. * Fix a misuse of free types when trying to infer the type of a subscript expression. * Fix a case of assertion failure when trying to resolve an overload from `never`. ### Native Code Generation * Fix dead store optimization issues caused by partial stores. --- ### Internal Contributors Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: David Cope <dcope@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com> --------- Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Vighnesh <vvijay@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: David Cope <dcope@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-03-15 23:37:39 +00:00
// Check if native execution is enabled
[[nodiscard]] bool isNativeExecutionEnabled(lua_State* L);
// Enable or disable native execution according to `enabled` argument
void setNativeExecutionEnabled(lua_State* L, bool enabled);
Sync to upstream/release/628 (#1278) ### What's new? * Remove a case of unsound `table.move` optimization * Add Luau stack slot reservations that were missing in REPL (fixes #1273) ### New Type Solver * Assignments have been completely reworked to fix a case of cyclic constraint dependency * When indexing, if the fresh type's upper bound already contains a compatible indexer, do not add another upper bound * Distribute type arguments over all type families sans `eq`, `keyof`, `rawkeyof`, and other internal type families * Fix a case where `buffers` component weren't read in two places (fixes #1267) * Fix a case where things that constitutes a strong ref were slightly incorrect * Fix a case where constraint dependencies weren't setup wrt `for ... in` statement ### Native Codegen * Fix an optimization that splits TValue store only when its value and its tag are compatible * Implement a system to plug additional type information for custom host userdata types --- ### Internal Contributors Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Alexander McCord <amccord@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Vighnesh Vijay <vvijay@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com> --------- Co-authored-by: Aaron Weiss <aaronweiss@roblox.com> Co-authored-by: Andy Friesen <afriesen@roblox.com> Co-authored-by: Vighnesh <vvijay@roblox.com> Co-authored-by: Aviral Goel <agoel@roblox.com> Co-authored-by: David Cope <dcope@roblox.com> Co-authored-by: Lily Brown <lbrown@roblox.com> Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
2024-05-31 20:18:18 +01:00
// Given a name, this function must return the index of the type which matches the type array used all CompilationOptions and AssemblyOptions
// If the type is unknown, 0xff has to be returned
using UserdataRemapperCallback = uint8_t(void* context, const char* name, size_t nameLength);
void setUserdataRemapper(lua_State* L, void* context, UserdataRemapperCallback cb);
using ModuleId = std::array<uint8_t, 16>;
// Builds target function and all inner functions
CompilationResult compile(lua_State* L, int idx, unsigned int flags, CompilationStats* stats = nullptr);
CompilationResult compile(const ModuleId& moduleId, lua_State* L, int idx, unsigned int flags, CompilationStats* stats = nullptr);
CompilationResult compile(lua_State* L, int idx, const CompilationOptions& options, CompilationStats* stats = nullptr);
CompilationResult compile(const ModuleId& moduleId, lua_State* L, int idx, const CompilationOptions& options, CompilationStats* stats = nullptr);
// Generates assembly for target function and all inner functions
std::string getAssembly(lua_State* L, int idx, AssemblyOptions options = {}, LoweringStats* stats = nullptr);
Sync to upstream/release/576 (#928) * `ClassType` can now have an indexer defined on it. This allows custom types to be used in `t[x]` expressions. * Fixed search for closest executable breakpoint line. Previously, breakpoints might have been skipped in `else` blocks at the end of a function * Fixed how unification is performed for two optional types `a? <: b?`, previously it might have unified either 'a' or 'b' with 'nil'. Note that this fix is not enabled by default yet (see the list in `ExperimentalFlags.h`) In the new type solver, a concept of 'Type Families' has been introduced. Type families can be thought of as type aliases with custom type inference/reduction logic included with them. For example, we can have an `Add<T, U>` type family that will resolve the type that is the result of adding two values together. This will help type inference to figure out what 'T' and 'U' might be when explicit type annotations are not provided. In this update we don't define any type families, but they will be added in the near future. It is also possible for Luau embedders to define their own type families in the global/environment scope. Other changes include: * Fixed scope used to find out which generic types should be included in the function generic type list * Fixed a crash after cyclic bound types were created during unification And in native code generation (jit): * Use of arm64 target on M1 now requires macOS 13 * Entry into native code has been optimized. This is especially important for coroutine call/pcall performance as they involve going through a C call frame * LOP_LOADK(X) translation into IR has been improved to enable type tag/constant propagation * arm64 can use integer immediate values to synthesize floating-point values * x64 assembler removes duplicate 64bit numbers from the data section to save space * Linux `perf` can now be used to profile native Luau code (when running with --codegen-perf CLI argument)
2023-05-12 18:50:47 +01:00
using PerfLogFn = void (*)(void* context, uintptr_t addr, unsigned size, const char* symbol);
void setPerfLog(void* context, PerfLogFn logFn);
} // namespace CodeGen
} // namespace Luau