How Does a Spacecraft Gain Speed During a Gravity Assist?

I understand the basic idea of a spacecraft passing near a planet to change its path, but I’m still puzzled by the “gain speed” part. If gravity pulls the spacecraft in and then back out, why doesn’t it leave with the same speed it had on arrival?

Is the key difference that the planet itself is moving around the Sun? I’d like to understand how a flyby can change a spacecraft’s speed relative to the Sun without the spacecraft using its engine for a major burn.