If it's later and not sooner, those who will be expelled from Israel (the Jews) will be the ones who spent their entire life in Israel while those who will replace the Jews will be people who never stepped a foot in Israel.
And it can happen with a war or something, so it's cool, but I'm not sure if the new inhabitants can be considered "rightful owners".