Port is always masculine though, it only means port/harbour.

But it's not that important, gender does add some redundancy and some error-correction to a language. It's not necessary (as shown by English) but it has some use.

I guess he's thinking of "la porte" but the pronunciation isn't the same.