The Romans exiled most of the Jews from Palestine after the failed Jewish uprisings between 64 and 132 CE (encompassing multiple revolts). After that, even before the Edict of Milan in 313CE, the region became a major Christian center, but after the Christianization of the Roman Empire it was overwhelmingly Christian. When the Muslims invaded the region in the 7th century CE, there weren't a lot of Jews in the region, and it was majority Christian.
And just to tell you who the real enemies of the Jews were, when the First Crusade made its way to the Holy Land, the capture of Jerusalem in 1099 led to the savage butchery of Jews and Muslims in the city. After the loss of the Crusader kingdoms in the region, it was Turkish territory, and while the Ottomans weren't always terribly nice to Jews, they were a helluva lot better than the Christians, and there was a steady flow of Jews back into the area during the Ottoman period.
And there were most certainly people living in Palestine, as there was resistance to Jews who began building settlements. It was not, as later claims would have it, an empty land. Most of the owners of the land were absentee landlords, but that is not the same thing at all.
But no, the Muslims did not displace the Jews, that was the Romans, some 1700 or 1800 years before Jews began returning in significant numbers.