No. Empirical ideas require confirmation in the material world, not necessarily direct observation. An empirical idea is a direct description of reality. We can confirm it by seeing it, measuring it, detecting its effects, reproducing its behavior, or deducing it from reliable material evidence. Direct observation is one path to empirical truth, but it is not the only path.
The Oort Cloud shows the difference. It is a serious speculative idea because long-period comets seem to point toward a distant reservoir of icy bodies surrounding the solar system. The idea is reasonable, even likely, but the cloud itself has not been confirmed. We have not yet observed, measured, sampled, or otherwise verified the proposed reservoir. So, for now, it remains speculative.
Viruses show the other side. Scientists confirmed viruses before they could see them. In 1892, experiments showed that something smaller than bacteria could pass through filters and still cause disease. That was empirical confirmation by effect, not by sight. Later, electron microscopes allowed us to see viruses directly. The idea became empirical before direct observation because the material world was already pushing back in repeatable, measurable ways.