First, let me say that I am a newb and know very little about the sport of scuba diving.
I recently went through open water certification classes. The most physically difficult class for me was the first when they had us swim the 200 yds (any stroke, unlimited time) and tread water for 10 mins. If you can do that, you'll have met the physical requirements to get through the class.
However.. (this is where I recite someone else's opinion in the hope of giving you food for thought), like any other sport, proficiency in skills adds to the your safety and enjoyment. Do you need to be proficient at different kick techniques to dive? No, but as I understand it, you would enjoy yourself more and be more relaxed if you did. Do you need to be proficient at buoyancy? Again, it is something that I have heard lets you be in better control and keeps your from accidental ascents and descents (issues I am personally still working on).
I personally think that swimming profiency can help you be more comfortable particularly in the unlikely event that things go south on a dive. As an example, if you had to assist your buddy back to shore, strong swimming skills would definitely be a plus. Also, if you find yourself in a situation where you are unable to return to your exit point underwater and have to fight waves and outgoing tide, more swimming capacity is better than less. These situations are definitely unlikely scenarios but it only seems prudent to be prepared to save your own bacon (and that of your buddy's) if the need ever arises.
HTH.