But all our assumptions or beliefs about what is possible and what can/would be observable is based on our, well, current understanding of reality.
Since generally speaking, humans know or understand a bit more today than they did yesterday, it seems likely that there are things that today we think we know but do not know or know incorrectly. Thus, our rules about what is observable may be flawed. We may be missing evidence of other civilizations and visitors.
Given the vastness of time and space and the fact that we DO know that there is a lot that we do not know (as opposed to not knowing what we do not know), it seems more reasonable to err on the side of we-probably-don't-know.
And the Fermi Paradox says that if it did, we would see evidence of its effects... but we don't, which imples FTL isn't even remotely possible.