In the end, none of this "belongs" to a country.

So, if object X is being, or would very likely be, damaged by being in country Y, I believe country Y has lost all right to the ownership of object X

You are arguing against the basic premise of a nation state, which is that a country has sovereign and complete rights over the geographical area and people generally acknowledged to belong to it currently. Since oil deposits formed in Saudi Arabia before the creation of their country, does oil become world heritage?

The international community can only show their disagreement through diplomatic and economic pressures. This is not only sound theory, but ensures such strong measures are only taken on solid grounds, not on a vague principle like you mentioned above. Thankfully, military measures have largely decreased, despite Iraq and Afghanistan.

Also, what has historically spurred military action is the lure of land-based riches and in rare cases, tragic oppression of human rights.

It's extremely presumptous to dispossess a nation of some resource from afar. What if a democratically elected government decides to raze historical monuments for some great material benefit? Or what if most citizens are happy with an existing monarchy and it makes such a decision?