I've never heard of anyone complaining of being too warm. Since you're someone who already gets cold easily and planning on 3 dives per day, I'd definately take the 5mm suit. You may be hot on the boat, and during a dive might have to cool off by pulling the neck open and letting some water in, but that's manageable.
The flip side is that if you go with the 3mm, which might be enough, but it's cool during your surface intervals, you'll wish you had the warmer suit, and there won't be an easy remedy.
Many of the locals dive in 5mm suits and more, so you won't be an oddball, and the only drawback is the extra ballast you'll need to carry.