I know that for me, the older I get, the harder it is to have "friends" or confidants. I am not a person that will share secrets or things like that, but I am very friendly and open-minded about many things, so its easy for me to accept people with what I would consider "flaws". I also find that I have a sense of humor that helps me be more accepting sometimes.
Nobody is perfect and if I do not enjoy someones company, I just do not look to have it. Actually I have more "online" good friendships than I do in my daily life. I dont have just that one friend that we do it all with, but for me I dont feel the need because I am content with having different friends and seeing them from time to time. I enjoy that more I think than having a "best" friend to do everything with.
Maybe because when I was young we moved around a lot and all the good people were taken ha ha. My guess is with the fact that you move around lot maybe part of you that you have not realized in the back of your mind, tries not to get attached to people because you know you will not have them in your daily life forever?
Im not sure exactly what things you mean drive you crazy because some people have a lower or higher tolerance in such things, but if it is very unbearable I can understand you wanting not to be around such people.
I have had a friend, for intance, that I knew from grade school and although we got along very well she becane very clingy and wanted me to contact her every single day even for a minute, which was impossible some days. When I did not, that caused her to get really agitated and one fine day she complained so much that I really got offended and we argued. Now normally, I forgive and forget as they say, but she was so upset she never spoke to me again. Others do and others dont. Her tolerance was low, where as someone else would try to understand and life would go on.
What I am trying to say is you are not terrible because you find you are not compatable with some people. Its not easy to "click" with people the older you get, especially if you move around a lot. It is a great thing that you have your husband that you are "friends" with, many women are not so lucky. Maybe all you need is each other