16.4 — Passing and returning std::vector, and an introduction to move semantics

When we need to pass a std::vector to a function, we pass it by (const) reference so that we do not make an expensive copy of the array data.

Therefore, you will probably be surprised to find that it is okay to return a std::vector by value in some cases.

Say whaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaat?

Copy semantics

Consider the following program:

#include <iostream>
#include <vector>

int main()
{
    std::vector arr1 { 1, 2, 3, 4, 5 }; // list construct arr1
    std::vector arr2 { arr1 };          // copy construct arr2 using arr1

    arr1[0] = 6; // We can continue to use arr1
    arr2[0] = 7; // and we can continue to use arr2

    return 0;
}

When arr2 is initialized with arr1, the copy constructor of std::vector is called, which copies arr1 into arr2. Making a copy is the only reasonable thing to do in this case, as both arr1 and arr2 live on independently and are used later in the program.

The term copy semantics refers to the rules that determine when copies of objects are made. When we say a type supports copy semantics, we mean that objects of that type are copyable, because the rules for making such copies have been defined. When we say copy semantics are being invoked, that means we’ve done something that will make a copy of an object.

For class types, copy semantics are typically implemented via the copy constructor (and copy assignment operator), which defines how objects of that type are copied. Typically this results in each data member of the class type being copied. In the prior example, the statement std::vector arr2 { arr1 }; invokes copy semantics, resulting in a call to the copy constructor of std::vector, which then makes a copy of each data member of arr1 into arr2. The end result is that arr1 is equivalent to (but independent from) arr2.

When copy semantics is not optimal

Now consider this related example:

#include <iostream>
#include <vector>

int main()
{
    std::vector arr2 { std::vector { 1, 2, 3, 4, 5 } }; // list construct a temporary std::vector, then construct arr2 using the temporary

    // We can't access the temporary std::vector
    arr2[0] = 7; // we can continue to use arr2

    return 0;
}

When arr2 is initialized this time, it is being initialized using a temporary std::vector. The usual thing to do here would be the same as in the previous example: use copy semantics and make a (potentially expensive) copy of the initializer. That way arr2 gets its own copy of the data, and doesn’t have to worry about what happens to the initializer beyond that point.

However, unlike the prior case where the initializer was an lvalue that could be used in future statements, in this case the initializer is an rvalue that will be destroyed at the end of the initialization expression.

In such cases, making a copy of the data and then destroying the original data is suboptimal.

Introduction to move semantics

Instead, what if there was a way for arr2 to “steal” the temporary object’s array data instead of copying it? arr2 would then be the new owner of the array data, and no copy of the data would need to be made.

When ownership of data is transferred from one object to another, we say that data has been moved. Most data can’t be moved, but when it can be, the cost of a move is typically just a few pointer assignments. When the data is large (and thus expensive to copy), moving is much cheaper than copying!

The term move semantics refers to the rules that determine when the data from one object may be moved to another object instead of copied. Move semantics is only available to class types.

When a class type object is moved, all data members that can be moved are moved to the new owner. Any data member that can’t be moved is copied instead.

In the worst case, if no data members can be moved, then the result of a move is exactly the same as a copy.

When move semantics is invoked

Normally, when an object is initialized or assigned using an object of the same type, copy semantics are used (unless the copy is elided altogether).

However, move semantics will be used instead of copy semantics when all of the following are true:

• The copy isn’t elided.
• The object’s type is a class type that supports move semantics.
• The object used for initialization or assignment is an rvalue (e.g. a temporary object). In the return statement of a function that returns by value, the returned object may also be an lvalue with automatic duration.

One of the neatest things about move semantics is that it is invoked automatically in eligible cases. We’ll illustrate this below.

However, the sad news is that not that many types support move semantics. std::vector and std::string are the two most commonly used types that do.

However, one of the neatest things about move semantics is that it is invoked automatically and transparently.

We’ll dig into how move semantics works in more detail (and how to make your class types move capable) in chapter 22.

Move semantics and const objects

Copy semantics does not modify the object being copied. We can copy a const or non-const object all the same.

Move semantics does require modifying the object being moved (as resources are transferred out of the object). This means move semantics can not be invoked on const objects.

Move semantics and return by value

In lessons 2.5 -- Introduction to local scope and 14.14 -- Introduction to the copy constructor, we discussed how return by value normally copies the return value into a temporary object that is returned to the caller. If this temporary object is then used to initialize a variable, another copy of the return value must be made. These copies may or may not be elided, depending on language standard and circumstances.

For the remainder of this section, we’ll assume no copies are elided.

Consider this example:

#include <iostream>
#include <vector>

std::vector<int> generate() // return by value
{
    return std::vector { 1, 2, 3, 4, 5 }; // constructs and returns a temporary
}

int main()
{
    std::vector arr2 { generate() };

    return 0;
}

First, let’s assume that std::vector only supports copy semantics. Within generate(), a temporary std::vector is constructed using the list constructor. Because this temporary dies at the end of the expression, it is not accessible by the caller. Instead, it is copied into another temporary in the scope of the caller. The temporary in the scope of the caller is then copied into arr2. Two expensive copies later, arr2 is ready to be used within the rest of main().

Since std::vector actually supports move semantics, let’s discuss what happens instead. Within generate(), a temporary std::vector is constructed using the list constructor. Because this temporary is an rvalue, it is moved into another temporary in the scope of the caller. Because the temporary in the scope of the caller is also an rvalue, it is then moved into arr2. Two inexpensive moves later, arr2 is ready to be used.

The only difference between these two examples is whether the type supports move semantics or not!

Now let’s look at a permutation of the above example:

#include <iostream>
#include <vector>

std::vector<int> generate() // return by value
{
    std::vector arr1 { 1, 2, 3, 4, 5 }; // constructs a local variable (with automatic duration)
    return arr1; // now returns the local variable
}

int main()
{
    std::vector arr2 { generate() };

    return 0;
}

In this version, the return statement no longer constructs and returns a temporary. Instead, we’re returning local variable arr1, which is an lvalue. Normally we can’t move lvalues, because those lvalues may be accessed again later. If that were the case here, we’d need to copy arr1 into the temporary in the scope of the caller, which could then be moved into arr2.

But notice that an lvalue with automatic duration used in a return statement of a function actually will be destroyed at the end of the expression (when the function returns). Therefore, this is a special case where move semantics will be used with an lvalue object. Accordingly, arr1 is moved into the temporary in the scope of the caller, which is then moved into arr2.

In other words, the result is the same as the prior example.

Now let’s take a look at two similar case where move semantics cannot be employed:

#include <iostream>
#include <vector>

std::vector<int> generate() // return by value
{
    static std::vector arr1 { 1, 2, 3, 4, 5 }; // now a static local
    return arr1; // now returns the local variable
}

std::vector<int> doSomething(const std::vector<int>& arr3)
{
    return arr3;
}

int main()
{
    std::vector arr2 { generate() };
    std::vector arr4 { doSomething(arr2) };

    return 0;
}

In this example, arr1 is now a static local variable (with static duration). Because arr1 is not destroyed when the function returns, arr1 can’t be moved.

Relatedly, in function doSomething(), arr3 is a const std::vector<int>&. Because the object bound to arr3 is not destroyed when the function returns, arr3 can’t be moved. Also arr3 is const, and const objects can’t be moved from.

How does move semantics change how we should pass and return values?

This section is adapted from C++ Core Guidelines.

Prior to move semantics, this was the recommendation:

For classes that support move semantics, we add a new column:

Wait, wait, wait. Expensive-to-copy types shouldn’t be passed by value, but if they are move-capable they can be returned by value?

Correct.

This following discussion is optional, but may help you understand why this is the case.

One of the most common things we do in C++ is pass a value to some function, and get a different value back. When the passed values are class types, that process involves 4 steps:

1. Construct the value to be passed.
2. Actually pass the value to the function.
3. Construct the value to be returned.
4. Actually pass the return value back to the caller.

Here’s an example of the above process using std::vector:

#include <iostream>
#include <vector>

std::vector<int> doSomething(std::vector<int> v2)
{
    std::vector v3 { v2[0] + v2[0] }; // 3 -- construct value to be returned to caller
    return v3; // 4 -- actually return value
}

int main()
{
    std::vector v1 { 5 }; // 1 -- construct value to be passed to function
    std::cout << doSomething(v1)[0] << '\n'; // 2 -- actually pass value

    std::cout << v1[0] << '\n';

    return 0;
}

First, let’s assume std::vector is not move-capable. In that case, the above program makes 4 copies:

1. Constructing the value to be passed copies the initializer list into v1
2. Actually passing the value to the function copies argument v1 into function parameter v2.
3. Constructing the value to be returned copies the initializer into v3
4. Actually returning the value to the caller copies v3 back to the caller.

Now let’s talk about how we might optimize the above. We have many tools are our disposal here: pass by reference or address, elision, move semantics, and out parameters.

We can’t optimize copies 1 and 3 at all. We need a std::vector to pass to the function, and we need a std::vector to return -- these objects have to be constructed. std::vector is an owner of its data, so it necessarily makes a copy of its initializer.

What we can affect is copies 2 and 4.

Copy 2 is made because we’re passing by value from the caller to the called function. What other options do we have?

• Can we pass by reference or address? Yes. We are guaranteed that the argument will exist for the entire function call -- the caller does not have to worry about the passed object unexpectedly going out of scope.
• Can this copy be elided? No. Elision only works when we’re making a redundant copy or move. There’s no redundant copy or move here.
• Can we use an out parameter here? No. We’re passing a value to the function, not getting a value back.
• Can we use move semantics here? No. The argument is an lvalue. If we moved data from v1 to v2, v1 would become an empty vector, and subsequently printing v1[0] would lead to undefined behavior.

Clearly pass by const reference is our best option here, as it avoids the copy, avoids null pointer issues, and works with both lvalue and rvalue arguments.

Copy 4 is made because we’re passing by value from the called function back to the caller. What other options do we have here?

• Can we return by reference or address? No. The object we’re returning is created as a local variable inside the function, and will be destroyed when the function returns. Returning a reference or pointer will result in the caller receiving a dangling pointer or reference.
• Can this copy be elided? Yes, possibly. If the compiler is smart, it will realize that we’re constructing an object in the scope of the called function and returning it. By rewriting the code (under the as-if rule) so that v3 is constructed in the scope of the caller instead, we can avoid the copy that would otherwise be made when returning. However, we are reliant upon the compiler realizing it can do this, so it is not guaranteed.
• Can we use an out parameter here? Yes. Instead of constructing v3 as a local variable, we can construct an empty std::vector object in the scope of the caller, and pass it to the function by non-const reference. The function can then fill this parameter with data. When the function returns, this object will still exist. This avoids the copy, but also has some significant downsides and constraints: ugly calling semantics, doesn’t work with objects that don’t support assignment, and it is challenging to write such functions that can work with both lvalue and rvalue arguments.
• Can we use move semantics here? Yes. v3 is going to be destroyed when the function returns, so instead of copying v3 back to the caller, we can use move semantics to move its data to the caller, avoiding the copy.

Elision is the best option here, but whether it happens is out of our control. The next best option for move-capable types is move semantics, which can be used in cases where the compiler doesn’t elide the copy. And for move-capable types, move semantics is invoked automatically when returning by value.

To summarize, for move-capable types, we prefer to pass by const reference, and return by value.

There’s just one more complexity here: an move-capable object that is passed by const reference can’t take advantage of move semantics when the same object is returned by value. So if you’re passing AND returning the same object, it’s best to use an out parameter.