Dude. I cant remember if you had a speed 600bb or just an ordinary speed 600? the bb is ok to 12v but the ordinary ones only to 7.2 or 9.6. None of them draw more that 10 or 12 amps a best efficiency. Your ESc must be shot. I think you had a 10Ahr 6v batt sla type? get a viper marine 25A ESc and this will be more than enough for the motor, without anything getting hot. (the motor will get warm with proplonged thrashing) I would buy a 30,35 and 40mm 2 blade ( NOT x-pitch) props to try out. Generally speaking these 2-blade props are better at higher revs. Get perhaps a few 3-blade plastic ones to try as well, they are cheap enough,same sizes. The only way to decrease weight is to use a NiMh pack, this will be half the weight of yer SLA batteries. I would stick to the lead ones for now, as high capacity NiMh cells are reasonably dear, and you have a boat big enough to carry a big batt, and a big battery already.
Over running the 600 on 12v will knacker it to short order. A definite solution may be to swap the 600 for an MFA 850, running 12v on a 50mm 2-blade prop, the combination MFA recomend. This is a proven package for boats your size. About £25 for a motor. runs at about 10A current draw..you would need a 12v battery, or get another 6v one. Ashley.