Added a getter for the Lexer's private `offset` to track its cursor
position in the buffer.
This helps me index tokens by buffer address in a project where I'm
rendering Luau code with [Dear ImGui](https://github.com/ocornut/imgui).
Would love this merged so I can use official Luau releases again!
Co-authored-by: Adrian Duermael <adrien@cu.bzh>
## What's new
This update brings improvements to the new type solver, roundtrippable
AST parsing mode and closes multiple issues reported in this repository.
* `require` dependency tracing for non-string requires now supports `()`
groups in expressions and types as well as an ability to type annotate a
value with a `typeof` of a different module path
* Fixed rare misaligned memory access in Compiler/Typechecker on 32 bit
platforms (Closes#1572)
## New Solver
* Fixed crash/UB in subtyping of type packs (Closes#1449)
* Fixed incorrect type errors when calling `debug.info` (Closes#1534
and Resolves#966)
* Fixed incorrect boolean and string equality comparison result in
user-defined type functions (Closes#1623)
* Fixed incorrect class types being produced in user-defined type
functions when multiple classes share the same name (Closes#1639)
* Improved bidirectional typechecking for table literals containing
elements that have not been solved yet (Closes#1641)
## Roundtrippable AST
* Added source information for `AstStatTypeAlias`
* Fixed an issue with `AstTypeGroup` node (added in #1643) producing
invalid AST json. Contained type is now named 'inner' instead of 'type'
* Fixed end location of the `do ... end` statement
---
Internal Contributors:
Co-authored-by: Hunter Goldstein <hgoldstein@roblox.com>
Co-authored-by: Talha Pathan <tpathan@roblox.com>
Co-authored-by: Varun Saini <vsaini@roblox.com>
Co-authored-by: Vighnesh Vijay <vvijay@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
We have a bunch of small grammatical nits and slightly awkward phrasings
present in our existing markdown files. This is a small pass over all of
them to fix those, and to provide some additional updated information
that has become more clear over time (like additional users of Luau, or
our leveraging something akin to the Minus 100 Points philosophy for
evaluating RFCs).
---------
Co-authored-by: Varun Saini <61795485+vrn-sn@users.noreply.github.com>
Ubuntu 20.04 LTS is reaching the end of its standard support period.
GitHub runners are going to stop providing images for it soon.
This PR updates all workflows to use 22.04 instead of 20.04.
While this will impact the compatibility of prebuilt release binaries on
20.04, users of those systems can always build Luau from source.
`coverage` workflow is also updating from clang++10 to clang++ (defaults
to clang++14 today).
Line coverage with this switch drops from 89.65% to 87.45%, function
coverage remains 91.26%.
As a bonus, we now get branch coverage information.
This change introduces a flag (`LuauUserTypeFunTypeofReturnsType`) that,
when enabled, sets `__type` on the type userdata's metatable to "type".
This behaviour was described in the user-defined type function RFC
(https://rfcs.luau.org/user-defined-type-functions.html), but seems to
have been missed; this change implements that behaviour.
Currently this does not change `typeof(t) == 'type'` emitting an unknown
type warning as I don't trust myself to implement it due to my general
lack of C++ knowledge; this can be worked on later.
# General
- Additional logging enabled for fragment autocomplete.
## Roundtrippable AST
- Add a new `AstNode`, `AstGenericType`
- Retain source information for `AstExprTypeAssertion`
## New Type Solver
- New non-strict mode will report unknown symbol errors, e.g
```
foo = 5
local wrong1 = foob <- issue warning
```
- Fixed a bug where new non-strict mode failed to visit large parts of
the program.
- We now infer the types of unnanotated local variables in statements
with multiple assignments, e.g. `local x: "a", y, z = "a", f()`
- Fixed bugs in constraint dispatch ordering.
- Fixed a bug that caused an infinite loop between `Subtyping`,
`OverloadResolution`, and `Type Function Reduction`, by preventing calls
to `Type Function Reduction` being re-entrant.
- Fixed a crash in bidirectional type inference caused by asserting read
and write properties on a type that was readonly.
## Runtime
- Fix a stack overflow caused by `luaL_checkstack` consuming stack space
even if the function fails to reserve memory.
- Using '%c' with a 0 value in Luau string.format will append a '\0'.
Resolves https://github.com/luau-lang/luau/issues/1650
## Miscellaneous
- Miscellaneous small bugfixes for the new solver.
**Full Changelog**:
https://github.com/luau-lang/luau/compare/0.660...0.661
----
Co-authored-by: Ariel Weiss <aaronweiss@roblox.com>
Co-authored-by: Hunter Goldstein <hgoldstein@roblox.com>
Co-authored-by: Talha Pathan <tpathan@roblox.com>
Co-authored-by: Varun Saini <vsaini@roblox.com>
Co-authored-by: Vighnesh Vijay <vvijay@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
---------
Co-authored-by: Hunter Goldstein <hgoldstein@roblox.com>
Co-authored-by: Varun Saini <61795485+vrn-sn@users.noreply.github.com>
Co-authored-by: Alexander Youngblood <ayoungblood@roblox.com>
Co-authored-by: Menarul Alam <malam@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
# General
This release introduces initial work on a Roundtrippable AST for Luau,
and numerous fixes to the new type solver, runtime, and fragment
autocomplete.
## Roundtrippable AST
To support tooling around source code transformations, we are extending
the parser to retain source information so that we can re-emit the
initial source code exactly as the author wrote it. We have made
numerous changes to the Transpiler, added new AST types such as
`AstTypeGroup`, and added source information to AST nodes such as
`AstExprInterpString`, `AstExprIfElse`, `AstTypeTable`,
`AstTypeReference`, `AstTypeSingletonString`, and `AstTypeTypeof`.
## New Type Solver
* Implement `setmetatable` and `getmetatable` type functions.
* Fix handling of nested and recursive union type functions to prevent
the solver from getting stuck.
* Free types in both old and new solver now have an upper and lower
bound to resolve mixed mode usage of the solvers in fragment
autocomplete.
* Fix infinite recursion during normalization of cyclic tables.
* Add normalization support for intersections of subclasses with negated
superclasses.
## Runtime
* Fix compilation error in Luau buffer bit operations for big-endian
machines.
## Miscellaneous
* Add test and bugfixes to fragment autocomplete.
* Fixed `clang-tidy` warnings in `Simplify.cpp`.
**Full Changelog**:
https://github.com/luau-lang/luau/compare/0.659...0.660
---
Co-authored-by: Ariel Weiss <aaronweiss@roblox.com>
Co-authored-by: Hunter Goldstein <hgoldstein@roblox.com>
Co-authored-by: Talha Pathan <tpathan@roblox.com>
Co-authored-by: Varun Saini <vsaini@roblox.com>
Co-authored-by: Vighnesh Vijay <vvijay@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
---------
Co-authored-by: Hunter Goldstein <hgoldstein@roblox.com>
Co-authored-by: Varun Saini <61795485+vrn-sn@users.noreply.github.com>
Co-authored-by: Alexander Youngblood <ayoungblood@roblox.com>
Co-authored-by: Menarul Alam <malam@roblox.com>
This test fails due to a bad interaction when `FFlagLuauStoreCSTData` is
enabled, whilst `FFlagLexerFixInterpStringStart` is disabled.
The OSS test suite, when run with `--fflags=true`, enables all
flags starting with a `Luau` prefix, but does not enable any other flag.
To resolve this, we explicitly enable both fflags for the failing
interpolated string test cases. As a consequence, we technically lose
the check that these tests pass when all flags are disabled.
## What's Changed
General performance improvements and bug fixes. `lua_clonetable` was
added too.
### General
## Runtime
- Improvements were made to Luau's performance, including a
`lua_clonetable` function and optimizations to string caching. Buffer
read/write operations were optimized for big-endian machines.
## New Solver
- Crashes related to duplicate keys in table literals, fragment AC
crashes, and potential hash collisions in the StringCache.
- We now handle user-defined type functions as opaque and track interior
free table types.
## Require By String
- Require-by-string path resolution was simplified.
**Full Changelog**:
https://github.com/luau-lang/luau/compare/0.658...0.659
---
Co-authored-by: Ariel Weiss <aaronweiss@roblox.com>
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Hunter Goldstein <hgoldstein@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
Co-authored-by: Varun Saini <vsaini@roblox.com>
Co-authored-by: Vighnesh Vijay <vvijay@roblox.com>
Co-authored-by: Yohoo Lin <yohoo@roblox.com>
---------
Co-authored-by: Hunter Goldstein <hgoldstein@roblox.com>
Co-authored-by: Varun Saini <61795485+vrn-sn@users.noreply.github.com>
Co-authored-by: Alexander Youngblood <ayoungblood@roblox.com>
a03c92095f6 #unflagged CLI-141149 Added lua_clonetable (#97902)
d66c43f36aa #flagged CLI-140903: Do not crash on duplicate keys in table literals (#97833)
8a2687f1690 #nonprod Luau: fix a test configuration issue breaking OSS CI. (#97878)
7add6d9dde9 #nojira CI-debugger revert #unflagged CLI-132461 Luau: refactor subtyping to also include the generic mapping it came up with. (#97872)
0c731fd3cc4 #flag-removal Clip `LuauNewSolverVisitErrorExprLvalues` (#96942)
2fe783b9615 #unflagged CLI-132461 Luau: refactor subtyping to also include the generic mapping it came up with. (#97803)
0cc41a0afae #unflagged CLI-141053 Luau buffer readbits/writebits implementation for big endian machines (#97826)
ae50bf04f99 #nonprod CLI-140027 Simplify require-by-string path resolution, fix bug when running CLI tools on unprefixed path (#97468)
77004599a6f #flag-removal Cleanup Luau VM and Compiler flags (#97799)
86777e269dd #flag-removal Remove LuauUserTypeFunPrintToError, LuauUserTypeFunNoExtraConstraint, LuauUserTypeFunUpdateAllEnvs, LuauUserTypeFunThreadBuffer and LuauUserTypeFunExportedAndLocal (#97624)
349b133fdc4 #nojira CI-debugger revert #unflagged CLI-132461 Luau: refactor subtyping to also include the generic mapping it came up with. (#97759)
2e820646ffa #flagged CLI-139615: Treat user defined type functions as opaque in eqSatSimplify (#96897)
c0845205e37 #flag-removal CLI-140688 Clean up `FFlagLuauIntersectNormalsNeedsToTrackResourceLimits` as `true`. (#97719)
3e07876a043 #flagged CLI-140571: Track interior free table types generated during constraint solving (#97498)
199b558dc36 #unflagged CLI-132461 Luau: refactor subtyping to also include the generic mapping it came up with. (#95646)
fd9255f62d3 #nonprod CLI-140762 unittest for fragment ac crash (#97669)
e9c710e017f #flag-removal FFlagLuauStoreCommentsForDefinitionFiles (#93359)
b184b940d55 CLI-140702 #unflagged Fix most clang-tidy warnings in TypeChecker2 (#97604)
1fc857f1bb3 CLI-140489 #unflagged Fix a potential hash collision bug in StringCache. (#97539)
58f62f900a2 #nonprod Some new unit testing macros for Luau tests. (#97336)
ab43a354b25 #flag-removal Remove LuauVectorMetatable and LuauVectorDefinitionsExtra (#97528)
c4c28694390 #flagged CLI-140485 Do not retain the Def/RefinementKey arenas when retainFullTypeGraphs is false (#97422)
benchmark / callgrind (map[branch:main name:luau-lang/benchmark-data], ubuntu-22.04) (push) Has been cancelled
build / macos (push) Has been cancelled
build / macos-arm (push) Has been cancelled
build / ubuntu (push) Has been cancelled
build / windows (Win32) (push) Has been cancelled
build / windows (x64) (push) Has been cancelled
build / coverage (push) Has been cancelled
build / web (push) Has been cancelled
release / macos (push) Has been cancelled
release / ubuntu (push) Has been cancelled
release / windows (push) Has been cancelled
release / web (push) Has been cancelled
## What's Changed
### General
- Allow types of tables to diverge after using `table.clone` (fixes
#1617).
- Allow 2-argument vector.create in Luau.
- Fix a crash when suggesting autocomplete after encountering parsing
errors.
- Add lua_tolstringatom C API which returns the string length (whether
or not the atom exists) and which extends the existing lua_tostringatom
function the same way lua_tolstring/lua_tostring do.
- Luau now retains the DFGs of typechecked modules.
### Magic Functions Migration Note
We've made a change to the API used to define magic functions.
Previously, we had a set of function pointers on each `FunctionType`
that would be invoked by the type inference engine at the correct point.
The problem we'd run into is that they were all `std::function`s, we'd
grown quite a few of them, and Luau allocates tens of thousands of types
as it performs type inference. This adds up to a large amount of memory
for data that isn't used by 99% of types.
To slim things down a bit, we've replaced all of those `std::function`s
with a single `shared_ptr` to a new interface called `MagicFunction`.
This slims down the memory footprint of each type by about 50 bytes.
The virtual methods of `MagicFunction` have roughly 1:1 correspondence
with the old interface, so updating things should not be too difficult:
* `FunctionType::magicFunction` is now `MagicFunction::handleOldSolver`
* `FunctionType::dcrMagicFunction` is now `MagicFunction::infer`
* `FunctionType::dcrMagicRefinement` is now `MagicFunction::refine`
* `FunctionType::dcrMagicTypeCheck` is now `MagicFunction::typeCheck`
**Full Changelog**:
https://github.com/luau-lang/luau/compare/0.657...0.658
---
Co-authored-by: Andy Friesen <afriesen@roblox.com>
Co-authored-by: Ariel Weiss <aaronweiss@roblox.com>
Co-authored-by: Aviral Goel <agoel@roblox.com>
Co-authored-by: Hunter Goldstein <hgoldstein@roblox.com>
Co-authored-by: Talha Pathan <tpathan@roblox.com>
Co-authored-by: Varun Saini <vsaini@roblox.com>
Co-authored-by: Vighnesh Vijay <vvijay@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
benchmark / callgrind (map[branch:main name:luau-lang/benchmark-data], ubuntu-22.04) (push) Has been cancelled
build / macos (push) Has been cancelled
build / macos-arm (push) Has been cancelled
build / ubuntu (push) Has been cancelled
build / windows (Win32) (push) Has been cancelled
build / windows (x64) (push) Has been cancelled
build / coverage (push) Has been cancelled
build / web (push) Has been cancelled
release / macos (push) Has been cancelled
release / ubuntu (push) Has been cancelled
release / windows (push) Has been cancelled
release / web (push) Has been cancelled
During conflict resolution while transferring patches back and forth,
some conflict resolutions may cause differences that are harder to spot.
This process would have also cleaned up differences noted in
https://github.com/luau-lang/luau/pull/1621
benchmark / callgrind (map[branch:main name:luau-lang/benchmark-data], ubuntu-22.04) (push) Has been cancelled
build / macos (push) Has been cancelled
build / macos-arm (push) Has been cancelled
build / ubuntu (push) Has been cancelled
build / windows (Win32) (push) Has been cancelled
build / windows (x64) (push) Has been cancelled
build / coverage (push) Has been cancelled
build / web (push) Has been cancelled
release / macos (push) Has been cancelled
release / ubuntu (push) Has been cancelled
release / windows (push) Has been cancelled
release / web (push) Has been cancelled
## General
- Fix a parsing bug related to the starting position of function names.
- Rename Luau's `Table` struct to `LuaTable`.
## New Solver
- Add support for generics in user-defined type functions
([RFC](https://rfcs.luau.org/support-for-generic-function-types-in-user-defined-type-functions.html)).
- Provide a definition of `math.lerp` to the typechecker.
- Implement error suppression in `string.format`.
- Fixes#1587.
- Ensure function call discriminant types are always filled when
resolving `FunctionCallConstraint`.
---
Co-authored-by: Ariel Weiss <aaronweiss@roblox.com>
Co-authored-by: Hunter Goldstein <hgoldstein@roblox.com>
Co-authored-by: Talha Pathan <tpathan@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
benchmark / callgrind (map[branch:main name:luau-lang/benchmark-data], ubuntu-22.04) (push) Waiting to run
build / macos (push) Waiting to run
build / macos-arm (push) Waiting to run
build / ubuntu (push) Waiting to run
build / windows (Win32) (push) Waiting to run
build / windows (x64) (push) Waiting to run
build / coverage (push) Waiting to run
build / web (push) Waiting to run
release / macos (push) Waiting to run
release / ubuntu (push) Waiting to run
release / windows (push) Waiting to run
release / web (push) Waiting to run
Implement RFC: 2-component vector constructor. This includes 2-component
overload for `vector.create` and associated fastcall function, and its
type definition. These features are controlled by a new feature flag
`LuauVector2Constructor`. Additionally constant folding now supports two
components when `LuauVector2Constants` feature flag is set.
Note: this work does not include changes to CodeGen. Thus calls to
`vector.create` with only two arguments are not natively compiled
currently. This is left for future work.
benchmark / callgrind (map[branch:main name:luau-lang/benchmark-data], ubuntu-22.04) (push) Waiting to run
build / macos (push) Waiting to run
build / macos-arm (push) Waiting to run
build / ubuntu (push) Waiting to run
build / windows (Win32) (push) Waiting to run
build / windows (x64) (push) Waiting to run
build / coverage (push) Waiting to run
build / web (push) Waiting to run
release / macos (push) Waiting to run
release / ubuntu (push) Waiting to run
release / windows (push) Waiting to run
release / web (push) Waiting to run
To implement math.lerp without branches, we add SELECT_NUM which
selects one of the two inputs based on the comparison condition.
For simplicity, we only support C == D for now; this can be extended to
a more generic version with a IrCondition operand E, but that requires
more work on the SSE side (to flip the comparison for some conditions
like Greater, and expose more generic vcmpsd).
Note: On AArch64 this will effectively result in a change in floating
point
behavior between native code and non-native code: clang synthesizes
fmadd (because floating point contraction is allowed by default, and the
arch always has the instruction), whereas this change will use
fmul+fadd.
I am not sure if this is good or bad, and if this is a problem in C or
not.
Specifically, clang's behavior results in different results between X64
and AArch64 when *not* using codegen, and with this change the behavior
when using codegen is... the same? :)
Fixing this will require either using LERP_NUM instead and hand-coding
lowering, or exposing some sort of "quasi" MADD_NUM (which would
lower to fma on AArch64 and mul+add on X64).
A small benefit to the current approach is `lerp(1, 5, t)`
constant-folds the
subtraction. With LERP_NUM this optimization will need to be implemented
manually as a partial constant-folding for LERP_NUM.
A similar problem exists today for vector.cross & vector.dot. So maybe
this
is not something we need to fix, unsure.
benchmark / callgrind (map[branch:main name:luau-lang/benchmark-data], ubuntu-22.04) (push) Has been cancelled
build / macos (push) Has been cancelled
build / macos-arm (push) Has been cancelled
build / ubuntu (push) Has been cancelled
build / windows (Win32) (push) Has been cancelled
build / windows (x64) (push) Has been cancelled
build / coverage (push) Has been cancelled
build / web (push) Has been cancelled
release / macos (push) Has been cancelled
release / ubuntu (push) Has been cancelled
release / windows (push) Has been cancelled
release / web (push) Has been cancelled
# General
All code has been re-formatted by `clang-format`; this is not
mechanically enforced, so Luau may go out-of-sync over the course of the
year.
# New Solver
* Track free types interior to a block of code on `Scope`, which should
reduce the number of free types that remain un-generalized after type
checking is complete (e.g.: less errors like `'a <: number is
incompatible with number`).
# Autocomplete
* Fragment autocomplete now does *not* provide suggestions within
comments (matching non-fragment autocomplete behavior).
* Autocomplete now respects iteration and recursion limits (some hangs
will now early exit with a "unification too complex error," some crashes
will now become internal complier exceptions).
# Runtime
* Add a limit to how many Luau codegen slot nodes addresses can be in
use at the same time (fixes#1605, fixes#1558).
* Added constant folding for vector arithmetic (fixes#1553).
* Added support for `buffer.readbits` and `buffer.writebits` (see:
https://github.com/luau-lang/rfcs/pull/18).
---
Co-authored-by: Aaron Weiss <aaronweiss@roblox.com>
Co-authored-by: David Cope <dcope@roblox.com>
Co-authored-by: Hunter Goldstein <hgoldstein@roblox.com>
Co-authored-by: Vighnesh Vijay <vvijay@roblox.com>
Co-authored-by: Vyacheslav Egorov <vegorov@roblox.com>
