From ef47950f1987dc1ca487b232a56a3efade574d69 Mon Sep 17 00:00:00 2001 From: Arseny Kapoulkine Date: Tue, 1 Feb 2022 19:48:51 -0800 Subject: [PATCH] Update generalized-iteration.md --- rfcs/generalized-iteration.md | 30 ++++++++++++++++++++++++++++-- 1 file changed, 28 insertions(+), 2 deletions(-) diff --git a/rfcs/generalized-iteration.md b/rfcs/generalized-iteration.md index 6d14594f..41636483 100644 --- a/rfcs/generalized-iteration.md +++ b/rfcs/generalized-iteration.md @@ -6,11 +6,37 @@ Introduce support for iterating over tables without using `pairs`/`ipairs` as we ## Motivation -Why are we doing this? What use cases does it support? What is the expected outcome? +Today there are many different ways to iterate through various containers that are syntactically incompatible. + +To iterate over arrays, you need to use `ipairs`: `for i, v in ipairs(t) do`. The traversal goes over a sequence `1..k` of numeric keys until `t[k] == nil`, preserving order. + +To iterate over dictionaries, you need to use `pairs`: `for k, v in pairs(t) do`. The traversal goes over all keys, numeric and otherwise, but doesn't guarantee an order; when iterating over arrays this may happen to work but is not guaranteed to work, as it depends on how keys are distributed between array and hash portion. + +To iterate over custom objects, whether they are represented as tables (user-specified) or userdata (host-specified), you need to expose special iteration methods, for example `for k, v in obj:Iterator() do`. + +All of these rely on the standard Lua iteration protocol, but it's impossible to trigger them in a generic fashion. Additionally, you *must* use one of `pairs`/`ipairs`/`next` to iterate over tables, which is easy to forget - a naive `for k, v in tab do` doesn't work and produces a hard-to-understand error `attempt to call a table value`. + +This proposal solves all of these by providing a way to implement uniform iteration with self-iterating objects by allowing to iterate over objects and tables directly via convenient `for k, v in obj do` syntax, and specifies the default iteration behavior for tables, thus mostly rendering `pairs`/`ipairs` obsolete - making Luau easier to use and teach. ## Design -This is the bulk of the proposal. Explain the design in enough detail for somebody familiar with the language to understand, and include examples of how the feature is used. +In Lua, `for vars in iter do` has the following semantics (otherwise known as the iteration protocol): `iter` is expanded into three variables, `gen`, `state` and `index` (using `nil` if `iter` evaluates to fewer than 3 results); after this the loop is converted to the following pseudocode: + +``` +while true do + vars... = gen(state, index) + index = vars... -- copy the first variable into the index + if index == nil then break end + + -- loop body goes here +end +``` + +This is a general mechanism that can support iteration through many containers, especially if `gen` is allowed to mutate state. Importantly, the *first* returned variable (which is exposed to the user) is used to continue the process on the next iteration - this can be limiting because it may require `gen` or `state` to carry extra internal iteration data for efficiency. To work around this for table iteration to avoid repeated calls to `next`, Luau compiler produces a special instruction sequence that recognizes `pairs`/`ipairs` iterators and stores the iteration index separately. + +Thus, today the loop `for k, v in tab do` effectively executes `k, v = tab()` on the first iteration, which is why it yields `attempt to call a table value`. If the object defines `__call` metamethod then it can act as a self-iterating method, but this is not idiomatic, not efficient and not pure/clean. + +This proposal comes in two pars: general support for `__iter` metamethod and default implementation for tables without one. ## Drawbacks