Switching between 2 in parallel to 2 in series requires a 2 pole change over. Putting 3 in series and tapping off the voltages is just a wiper selecting the connection points, BUT the wiper must never connect two adjacent connections, and there is the probability that the battery nearest the ground (-ve) will flatten before its mates. Just leaving them in series and using the right ESC makes much more sense and will work. Not many rotary switches are designed to carry serious current, much less switch it.

Your proposed system would give you the problem of uneven battery charge states, which is undesirable since nice even recharging becomes a problem. This is the basic cause of the myth of battery memory which has circulated unimpeded for many years regarding NiCad packs.

If there is a need to restrict performance, just wire a nice big power resistor in series with the motor and use a switch to short it out when you want the full power back. The batteries get the care and attention they need, power restriction can happen when its needed and the resistor can be fairly cheap. For half speed, measure the resistance of the motor, take about 2/3 that value, thats the resistance wanted to give half the power to the motor, thus half the speed. Assume that the resistor will get the full supply voltage and use that to work out the power rating of the resistor. This will give a generous safety margin.