Good question Paul,
I think it makes a lot of difference - and while going over things like emergencies and the dive plan you have a good opportunity to gauge what your new buddy knows - and you do become more prepared.
I think that a major reason that a lot of people blisfully ignore this is that the general training for scuba is setup to induce a feeling of safety into people and get them out diving, most people do not really realize just how hostile the underwater environment is - if there is a massive failure, you are likely to die, that is just a fact. I think that when you start to really think about that, you also become very careful with how you behave and how you plan.
As far as new buddy - I often have an opportunity to talk to oterhs about them - since I know a lot of divers around here I sometimes get word of mouth as well...
If you meet someone out diving - just go over and ask how the last dive was - you'll be surprised what people will tell you - like they penetrated this little wreck and left their buddy waiting outside... (hmm, what do you think of that?).
I once dove with a guy who seemed ok, then at 130 on a wall - he decided that "same ocean" buddy system was how he was going to conduct the dive - fortunately he was 3rd leg on our buddy team so my regular buddy and I could continue our dive - when we got out of the water, he was already out and saying "Oh - I think I outswam you guys down there" so the guy checked out - but when we hit the water he did not pass...
Word comes around - he is no longer on our maillist - nobody wants to dive with him as the behavior is considered "accident waiting to happen" and nobody wants to be around for that - and when someone new comes up and says "I got invited in a dive by so and so" I say - don't go - the guy is dangerous - I'll take you diving there myself if you want to do that dive.
So - to ansswer your question - I don't think there are specific questions - you can look at how they put the rig together, it'll tell you if they are new or rusty... Then go on some easy dives first - no big deal right? If they are smart - they will be cool with you wanting to just feel them out as a buddy before doing major dives - and they may want to do the same thing...
Now - one thing I want to ask of you is - don't ever tell a newbie "NO", there are a lot of people out there who refuses to dive with newly trained divers - that's too bad - this is where your future dive buddies and friends come from - and like you - they are standing around and wondering who, when and where to dive...
Anyway - I think I'm going to go and dive...
Big T