FFS.
Ok, I suppose it depends on the definition of malice, but negligence can be a criminal act. You don't need to intend to actually cause harm, but if you knowingly act in a way that may cause harm, that's considered intent. Not internet :)
Specifically for libel sure, but I was using that as one of many examples where harmful speech is widely considered to be something that exists. I don't mean to say that specifically these are criminal libel machines.
My point is twofold: harmful speech exists and there are many examples of where it's codified in law even in the US with the first amendment. Secondly, intent is less than active malice, sound something you know may cause harm, the defence of "I didn't mean to" doesn't hold up.
So ok mostly I don't hold with the argument that there is no such thing as harmful speech. It's a very extreme position held by some people, but the law disagrees. Now different types have different bars in different jurisdictions. Solicitation of a crime for example is one where the bar for harm/intent is pretty low. Kind of implied by the crime part.
Safe harbour provisions shield the hoster from the actions of the people who make the material that's hosted. However with LLM output, they are the person making the material to be hosted so I don't see how the provisions apply.