Just look at the toxicity of elements we have not evolved around (in high quantities) e.g lead. Lead is not somehow inherently deadly to any form of life -- we just didn't evolve in the presence of high quantities of it.