I think what we do have evidence for is that you didn't read the paper, but I did, because it was interesting. From the paper:

Godel does no such thing. The incompleteness theorem says that some things can't be proven, and aren't computable, but every example of that *includes humans*. It's not a case that you can't build a computer and program in an axiomatic system that is consistent and can prove every statement with godel numbers, but that a human can prove a statement in that system that that computer can't prove. The human can't either. It's a statement about the limits of axiomatic mathematical systems.

There's no evidence anything in human thought falls under the realm of uncomputability. In fact, given that the brain is made up of neurons that are guaranteed to fire or not fire given specific conditions, electrical and chemical, there's plenty of evidence that it *must* be computable and algorithmic.

That is a terrible system, I'm surprised they do it that way. Banks are usually better about that. The only times I got a call from my bank that required me to prove who I was, it was either a returned call, and they mentioned the subject and that I had called, before they started verifying my identity, so I knew it was legit. Or the fraud alert people, and they could easily verify that they were who they said they were, because they asked about specific purchase attempts with the amount and location before they tried to verify my identity.

I did get one *actual* phishing call decades ago that made me absolutely crack up. The person on the other line said they were from "the bank." They didn't say which bank, just "the bank." Usually I immediately hang up on phishing, but that one made me want to engage a bit: I asked "which bank" and he answered, "your bank." At that point I just burst out laughing and the gig was up, so I hung up.

I had an instance of a work e-mail years ago, that was sent from a third-party contractor, that had so many red flags for very obvious phishing (including coming from outside the organization, wtf).

Where I work, we have a place to forward phishing emails so that IT can review it. I forwarded it there, and apparently so many other people did that a follow-up email had to be sent out that said, "we thank everyone for pointing out this e-mail as phishing, but we can confirm it's actually legit."

I think they learned the lesson from that, because it has not happened since that we got such a terrible email. I think my point is that overtraining may not work, but having a place to report phishing is a great idea. It only takes one person to report it, and then the IT department sends out a massive e-mail to warn everyone else about it, so it doesn't rely on them recognizing it (and anyone that already fell victim to it can report that they have, so action can be taken to minimize the damage). And in cases like you and I experienced, they can also do the opposite and confirm that it's real.

I agree, but go the other route for the comparison to humans and thought: people need to stop thinking that what we do when we "think" isn't algorithmic. Of course it is. We're not that special.

The models are trained on the same data, and they create their output based on the connections they made with all the previous data. When we ask it to generate "random" numbers, they're not any more random than when a human is asked to generate a random list of numbers. It's not purposefully encoding the information in the numbers because transmitting its love for owls is important to it, but the favorite animal information tokens are part of the seed made when it's generating those numbers.

Invariably, the second LLM that has been trained on the same data as the first will have the same data connections to those numbers. It's similar to how, when I was dating, I was filtering out anyone that added the information in the app that they had not been vaccinated for COVID. There's a *lot* of information associated with the type of person who was not only not vaccinated, but felt that they needed to state it. The information isn't contained in that assertion alone, but combined with the information already in my brain, it tells me a lot about their belief structure in things completely unrelated to vaccines and COVID. The LLM is doing that.

Might I suggest "Blind Lake" by Robert Charles Wilson. I read it when it first came out two decades ago, but it reads strangely similar to what we're seeing now, and the sinister edge is there, too.

Does this also say something more about the "coffee nap"?

Bingo. (no mod points today)

I wish I had mod points for this. My son-in-law works in this stuff and he's been frustrated about resistance to carbon-reduction efforts. The specific one he mentioned a while back I believe involved adding a (possibly calcium-containing) base to let a precipitate fall onto the sea bed sequestering the carbon. People were worried about sticking basic chemicals into the sea without realizing that reducing acidity itself was good in addition to carbon sequestration - that they're actually related.