No, space is not completely empty. Even the most “empty” regions of space contain tiny amounts of matter and energy. Think along the lines that gravity extends to all objects throughout the universe.
To break it down, let’s talk about the known forces and particles. To date, we’ve identified 17 fundamental particles and four forces. Space can be extremely sparse, and the average space between hydrogen atoms is likely about a meter. Between those atoms, the force of gravity exists everywhere all at once, so it will still exist. In between atoms, other particles are likely to be present too. For example, light, or photons, pass through empty space, and there are others too, like neutrinos.