Draft RFC: variadic generics

Issue by eddyb
Monday Oct 28, 2013 at 17:39 GMT

For earlier discussion, see rust-lang/rust#10124

This issue was labelled with: B-RFC in the Rust repository

The Problem

fn types need a reform, and being able to define a trait with a variable number of type parameters would help
working with functions which have a variable number of arguments is impossible right now (e.g. a generic bind method, f.bind(a, b)(c) == f(a, b, c)) and defining such functions may only be done (in a limited fashion) with macros
tuples also have similar restrictions right now, there is no way to define a function which takes any tuple and returns the first element, for example

The Solution: Part One

C++11 sets a decent precedent, with its variadic templates, which can be used to define type-safe variadic functions, among other things.
I propose a similar syntax, a trailing ..T in generic formal type parameters:

type Tuple<..T> = (..T); // the parens here are the same as the ones around a tuple.
// use => expansion
Tuple<> => ()
Tuple<int> => (int,)
Tuple<A, B, C> => (A, B, C)

The simple example above only uses ..T, but not T itself.
The question which arises is this: what is T? C++11 has a special case for variadic parameter packs, but we can do better.
We have tuples. We can use them to store the actual variadic generic type parameters:

// Completely equivalent definition:
type Tuple<..T> = T;
// Detailed expansion of (..T) from above:
Tuple<> => (..()) => ()
Tuple<int> => (..(int,)) => (int,)
Tuple<A, B, C> => (..(A, B, C)) => (A, B, C)

The Solution: Part Two

Now that we know the answer is "tuples", everything else is about extending them.
The prefix .. operator would expand a tuple type:

// in another tuple type:
type AppendTuple<T, U> = (..T, U);
type PrependTuple<T, U> = (U, ..T);
AppendTuple<PrependTuple<Tuple<B>, A>, C> => (A, B, C)
// in a fn type's parameter list:
type VoidFn<..Args> = fn(..Args);
VoidFn<A, B, C> => fn(A, B, C);
// in a variadic generic parameter list:
Tuple<..(A, B, C), ..(D, E, F)> => (A, B, C, D, E, F)

Then we can do the same thing with values:

// in another tuple:
(..(true, false), ..(1, "foo"), "bar") => (true, false, 1, "foo", "bar")
// in a function call:
f(..(a, b), ..(c, d, e), f) => f(a, b, c, d, e, f)

There's only one piece missing: we're still not able to define a function which takes a variable number of arguments.
For this, I propose ..x: T (where T is a tuple type) in a pattern, which can be used to "capture" multiple arguments when used in fn formal arguments:

// Destructure a tuple.
let (head, ..tail) = foo;
// This function has the type fn(int, A, B, C), and can be called as such:
fn bar(_: int, ..x: (A, B, C)) -> R {...}

A type bound for ..T (i.e. impl<..T: Trait>) could mean that the tuple T has to satisfy the bound (or each type in T, but that's generally less useful).

Examples:

// The highly anticipated Fn trait.
trait Fn<Ret, ..Args> {
    fn call(self, .._: Args) -> Ret;
}
impl Fn<Ret, ..Args> for fn(..Args) -> Ret {
    fn call(self, ..args: Args) -> Ret {
        self(..args)
    }
}
impl Fn<Ret, ..Args> for |..Args| -> Ret {
    fn call(self, ..args: Args) -> Ret {
        self(..args)
    }
}

// Cloning tuples (all cons-list-like algorithms can be implemented this way).
impl<Head, ..Tail: Clone> Clone for (Head, ..Tail) {
    fn clone(&self @ &(ref head, ..ref tail)) -> (Head, ..Tail) {
        (head.clone(), ..tail.clone())
    }
}
impl Clone for () {
    fn clone(&self) -> () {
        ()
    }
}

// Restricting all types in a variadic tuple to one type.
trait ArrayLikeTuple<T> {
    fn len() -> uint;
}
impl<T> ArrayLikeTuple<T> for () {
    fn len() -> uint {0}
}
impl<T, ..Tail: ArrayLikeTuple<T>> ArrayLikeTuple<T> for (T, ..Tail) {
    fn len() -> uint {
        1 + Tail::len()
    }
}

// And using that trait to write variadic container constructors.
impl<T> Vec<T> {
    fn new<..Args: ArrayLikeTuple<T>>(..args: Args) -> Vec<T> {
        let v: Vec<T> = Vec::with_capacity(Args::len());
        // Use an unsafe move_iter-like pattern to move all the args
        // directly into the newly allocated vector. 
        // Alternatively, implement &move [T] and move_iter on that.
    }
}

// Zipping tuples, using a `Tuple` kind and associated items.
trait TupleZip<Other>: Tuple {
    type Result: Tuple;
    fn zip(self, other: Other) -> Result;
}
impl TupleZip<()> for () {
    type Result = ();
    fn zip(self, _: ()) -> () {}
}
impl<A, ATail: TupleZip<BTail>, B, BTail: Tuple> TupleZip<(B, ..BTail)> for (A, ..ATail) {
    type Result = ((A, B), ..ATail::Result);
    fn zip(self @ (a, ..a_tail), (b, ..b_tail): (B, ..BTail)) -> Result {
        ((a, b), ..a_tail.zip(b_tail))
    }
}

Todo

better formal descriptions
figure out the details of pattern matching
more examples

There is one issue with my initial approach: a reference to a "subtuple" of a larger tuple could have different padding than the tuple type made out of those fields.

One solution I may have is a &(ref head, ..ref tail) pattern turning &(A, B, C, D) into &A and (&B, &C, &D), i.e. a subtuple of references instead of a reference of a subtuple.
But I have no idea how inference or generics would work with it.

This is probably a prerequisite for taking the Cartesian product of iterators without a macro, like in rust-itertools/iproduct! and rust-itertools/itertools#10.

This should be expanded to also support using the ".." operator on fixed size arrays (behaves equivalent to a tuple where the elements are all the same type and the arity matches the length of the array).

In combination with #174 this would make it much easier to deal with variadic arguments of the same type. Something which even C++ is still missing.

This would need to be compatible with range syntax.

How would one implement the cartesian product of two type tuples with this?

@gnzlbg

@gnzlbg what exactly do you have in mind, Product<(A, B), (X, Y)> = ((A, X), (A, Y), (B, X), (B, Y))?

@eddyb

@eddyb yes, but with variadics.

Having wrestled with c++11 templates and Rust macros, I really like this RFC. I hope it comes to fruition sooner rather than later.

Me too, I really wished this to come before 1.0 and solved tuple impl etc.

Unfortunately this syntax doesn't work, because the .. operator is already taken by range syntax. We could avoid this problem by using triple dots like C++. Which are also already taken (by inclusive range syntax), but only as infix operator and only in pattern syntax (though there were some requests to allow triple dots in expressions, too).

For the general idea: 👍, I was imagining variadic generics using pretty much the same approach. However, the devil is in the details that aren't yet worked out. C++ resolves expressions involving parameter pack expansion only after monomorphization; but I assume Rust would want to type-check the generic code? Wouldn't that require restrictions on where the unpacking operator is used? Also, consuming variadics in C++ usually involves a recursive function using template specialization. With this proposal, it seems like you'd need to create a trait for every variadic function, so that you can "specialize" using trait impls as in the Clone example?

Also, the ... operator in C++ allows "unpacking" complex expressions, not just directly unpacking the parameter pack. For example, in C++11 I can do:

template <typename... T> void consumer(T... t) { ... }

template <typename... T> void adapter(T... t)
{
   consumer(t.adapt()...);
   // expands to:
   // consumer(t1.adapt(), t2.adapt(), t3.adapt(), ...);
}

What's the easiest way to write the 3-line adapter function in Rust with this RFC? Do I have to write recursive Adapt implementations like the Clone impls in the example?

As for Vec::new: I think in cases where we don't actually need a type parameter pack, but just a variadic function with a single parameter type, it might be better to support something like this:
fn new(..elements: [T]) -> Vec<T> { ... }
This requires being able to pass unsized types as parameters - but that seems doable (and might already be planned?).

Why not the dereference operator?

type AppendTuple<T, U> = (*T, U);
f(*(a, b), *(c, d, e), f) => f(a, b, c, d, e, f);

@dgrunwald

@dgrunwald :

Also, consuming variadics in C++ usually involves a recursive function using template specialization.

In C++1y there are fold-expressions that allow you to do a lot of things without recursion:

template<typename... Args>
bool all(Args... args) { return (... && args); }

bool b = all(true, true, true, false);

but I assume Rust would want to type-check the generic code? Wouldn't that require restrictions on where the unpacking operator is used?

You can easily type check that all parameters in the parameter pack implement a given trait. If one needs more complicated type-checking a variadic function is likely not the best solution.

@gnzlbg

@gnzlbg that's C++1z actually.

@gnzlbg

The syntax between the value-level and type-level languages should be kept as close as possible. Already the + operator means the intersection of traits at the type level, even though * (or &) more accurately conveys a lattice meet.

@gnzlbg I don't think that form of duck-typing is compatible with Rust's trait-based generics.

Rather than having special fold syntax, it's cleaner to design some system compatible with HKTs for constraining the tuple's types, and then iterate over a tuple for fold any other operation:

fn fold_variadic<*T> (x: T) -> u32
    where *T: BitAnd<u32>
{
     x.iter().fold(!0u32, |b, a| b & a)
}

fold_variadic(0xFFF0, 0x0FFF) == 0x0FF0

In the above example, T is a tuple of variadic length, and * is a currying operator. This makes intuitive sense once you realize that type arguments are effectively pattern-matched in variadic generics. Consider the partial signature fold_variadic::<*T> and the invocation fold_variadic::<u32, u32, u32>, and notice how T is bound:

<*T> = <u32, u32, u32>
*T = u32, u32, u32

The curried/destructured tuple is matched to u32, u32, u32, so the tuple is (u32, u32, u32).

The interaction of a curried tuple with +, =, and :, as below, should also be specified.

where *T: BitAnd<u32>
// or
where *T = u32

A sensible semantic is distribution over the curried form. *T: BitAnd<u32> means every type in the tuple implements BitAnd<u32>, *T = u32 means every type in the (variable-length) tuple is u32.

This is a draft RFC, so I can pretty confidently say no

If you want optional arguments soon then implement them on nightly using procmacros

fn foo(..mandatory_args..) -> Foo { Foo { ..mandatory_args.. } }
pub struct Foo { ..mandatory_args.. }
impl FnOnce<()> for Foo { .. }
impl FnOnce<OptArg1> for Foo { .. }
impl FnOnce<OptArg2> for Foo { .. }
impl FnOnce<OptArg3> for Foo { .. }
impl FnOnce<OptArg4> for Foo { .. }

and invoke like foo(mandatory_args)(optarg3)

It'd even support different return types for different optional argument types.

@eddyb

Is anyone working on turning it into full RFC?
cc @eddyb @canndrew @Centril @nateling

There's been several attempts over the years and it doesn't seem like it's going to happen again any time soon, sorry.

I have a branch somewhere with some of the implementation details (that can be used internally by rustc even without a proper VG feature, in order to clean up the extern "rust-call" mess), but even that I don't know when I'll get back to.

Is there anyone else?

I don't think this will be implemented in 2018. It would still be pretty useful.

Related: https://poignardazur.github.io/2024/05/25/report-on-rustnl-variadics/

I'd expect some builder patters plus #3681 would be preferable for many if not most variadics use cases, ala MyFn { whatever args, .. }.go()

I suppose Iterator<Item = &dyn MyTrait> fits many of the remaining use cases, but if we'd some generic closure then maybe the compiler could derive a tuple trait from the base trait:

#[derive_tuple(MyTraitTuple)]
pub trait MyTrait { .. }

defines

trait MyTraitTuple {
    type Head: MyTrait;
    type Tail: MyTraitTuple;
    .. map fns using generic closures .. 
}

This would solve quite a lot of confusion when I have to implement few equivalent traits by look and feel with difference in tuples in types.

Code sample solvable by variadic generics

// Given by third-party crate A, cannot be changed
trait CustomTrait<Ctx: Clone> {
    fn read_data(ctx: Ctx) -> Self;
}

// Given by third-party crate B, cannot be changed
#[derive(Debug, Default, Clone)]
struct Inner;

// Given by third-party crate B, cannot be changed
#[derive(Debug, Clone, Copy)]
enum CommonCtx {
    Value,
}

// Given by third-party crate B, cannot be changed
#[derive(Debug, Clone, Copy)]
enum InnerCtx1 {
    Value,
}

// Given by third-party crate B, cannot be changed
#[derive(Debug, Clone, Copy)]
enum InnerCtx2 {
    Value,
}

// Given by third-party crate B, cannot be changed
impl CustomTrait<(CommonCtx, InnerCtx1, InnerCtx2)> for Inner {
    fn read_data(_ctx: (CommonCtx, InnerCtx1, InnerCtx2)) -> Self {
        Self::default()
    }
}

#[derive(Debug, Clone, Copy)]
enum OuterCtx {
    Value,
}

// Generic container
#[derive(Debug, Clone)]
struct Outer<V: Clone>(V);

impl<V, Ctx> CustomTrait<(CommonCtx, OuterCtx, Ctx)> for Outer<V>
where
    Ctx: Clone,
    // doesn't compile, because Ctx is tuple, flattening Ctx would solve the issue
    V: Clone + CustomTrait<(CommonCtx, Ctx)> + Default
{
    fn read_data(ctx: (CommonCtx, OuterCtx, Ctx)) -> Self {
        let (common_ctx, outer_ctx, inner_ctx) = ctx;
        if matches!(outer_ctx, OuterCtx::Value) {
            println!("matches!")
        }
        ctx_arg = (common_ctx, inner_ctx);
        // doesn't compile, because inner_ctx is tuple
        Self(<V>::read_data(ctx_arg))
    }
}

fn main() {
    let output = <Outer<Inner>>::read_data((CommonCtx::Value, OuterCtx::Value, (InnerCtx1::Value, InnerCtx2::Value)));
    println!("{output:#?}");
}

Python's standard document: https://peps.python.org/pep-0646/

Along with this, I'd suggest new syntax to check all types except one in the generic.
Eg.

impl<..T> MultidirectionalMap<..T, RandomState> {
   fn access_random_member<..T>(x: T in ..T) -> (..T - T in ..T) {
       // snip
   }

Hey, just to clarify — based on the blog post (https://poignardazur.github.io/2021/01/30/variadic-generics/), the plan is to have something like

fn unwrap_all<...Ts>(options: ...Option<Ts>) -> (...Ts)

and not something like

impl<...Ts> UnwrapAll for (Option<...Ts>)

right? (found in the survey results)

The impl<...Ts> UnwrapAll for (...Option<Ts>) form feels much less ambiguous and more logical to me. In the survey’s example, it’s really easy to confuse Option<...Ts> with Option<(...Ts)>. On top of that, the former makes it look like Option itself takes variadic template parameters — which is obviously not the case.

That’s why ...Option<Ts> feels more consistent and intuitive.

How about an optimisation focused based approach instead?

This should produce exactly the same code as requestions from variadic generic functions:

#[inline(always)]
fn variadic_generic_function(xs: &[&dyn Trait]) { for x in xs { .. } }

variadic_generic_function([&foo, &bar, &baz])

First &foo, &bar, &baz each get cast to &dyn Trait. Next variadic_generic should be inlined. Next the loop should be unrolled. And finally unnecessary vtable indirection should be removed.

If this were not true, then rust might require some #[inline(early)] directive, so that variadic_generic can be inlined early enough so the other optimisations work.

Abstraction over tuples might still make sense inside types, but this approach makes more sense for functions.

Abstraction over tuples might still make sense inside types, but this approach makes more sense for functions.

I disagree, dyn Trait is very limiting since a lot of useful traits are not dyn compatible, e.g. Hash (though there are much more complex traits than just Hash that I'd want to use with tuples, e.g. traits with associated types that depend on the tuple's field type.).

That said, dyn Trait is a useful workaround when your code happens to easily fit that pattern.

Add alternative APIs for arbitrary-length link creation

Navigation Menu

Repository navigation

Draft RFC: variadic generics #376

The Problem

The Solution: Part One

The Solution: Part Two

Examples:

Todo

47 remaining items

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Draft RFC: variadic generics #376

Description

The Problem

The Solution: Part One

The Solution: Part Two

Examples:

Todo

Activity

eddyb commented on Oct 9, 2014

alexchandel commented on Jan 23, 2015

Diggsey commented on Jan 24, 2015

alexchandel commented on Feb 4, 2015

gnzlbg commented on Feb 27, 2015

eddyb commented on Feb 27, 2015

gnzlbg commented on Feb 27, 2015

goertzenator commented on Feb 27, 2015

oblitum commented on Feb 27, 2015

dgrunwald commented on Feb 27, 2015

alexchandel commented on Mar 3, 2015

gnzlbg commented on Mar 3, 2015

oblitum commented on Mar 4, 2015

alexchandel commented on Mar 5, 2015

47 remaining items

Kixiron commented on Mar 29, 2021

burdges commented on Mar 29, 2021

Randl commented on Apr 1, 2021

eddyb commented on Apr 30, 2021

86maid commented on Jan 30, 2024

jasal82 commented on Apr 18, 2024

kirawi commented on Aug 23, 2024

burdges commented on Aug 24, 2024

eirnym commented on Apr 20, 2025

eirnym commented on Apr 20, 2025

scimind2460 commented on May 20, 2025

ThalusA commented on Sep 23, 2025

burdges commented on Oct 7, 2025

programmerjake commented on Oct 7, 2025

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions