Yes. Definitely a port problem or an issue with your firewall settings, assuming you use a router. I experienced the same thing whenever I installed a router for the first time. One server would have negative ping, the other showed it had 266 players online, or I could only successfully and flawlessly play in Server B but not A.
I messed with the damn thing (router) for hours, trying out all the configurations Google search results gave to me. In the end, I simply disabled the firewall, and it started working fine.
_________________
Serena Voi: An adorable little Halfling.
Brizthara D'ulov: A spoiled, loud-mouthed priestess.