Suspect Ignition Switch problem, Voltage you hardly ever have enough Options

[lsimburger]

I suspect I have a starter ignition switch problem. I installed a new 1.5 hp starter and it worked fine, for awhile. Now it starts very rarely on it's own, almost always after I give the starter a light tap then I get in the car it starts most of the time.
Here's what I have checked:
1) voltage at battery 11 volts not good
2) Ground strap checks good
3) between alternator wire (the smaller red wire at the starter) and ground .5 volts (problem with alternator? could this be draining my battery?)
4) between yellow and black wires (at starter) I get 10.3 - 10.7 volts with ignition in off position, my thinking is this should make the starter run full time so I think it is a trickle output, no way to test if it is only a trickle output that I know of. With ignition on start I get the same voltage.

I am finding myself second guessing my self with these strange readings. I suspect two things one is the alternator may be draining the battery, low voltage would make it hard to start. I also suspect the ignition switch. I am hoping someone can explain to me some ways of diagnosing what is going on so I can get this fixed!
Thanks in advance!