Well, whether Jesus had brothers or not can never be exactly known, but honestly, how much sense does it make for Joseph not to want to start nailing (haw haw) his wife? I'm sure they had plenty of sweet lovin'. Just kinda follows, I guess.