The only thing that will work will be to ban advertising to children who are supposed to be focused on schoolwork. In particular, to fine the companies being promoted in the ads.

Imagine how much he could have accomplished in 7!" - Doom

  Curiously enough, not observing the sabbath was punishable by death in Levitical law. Being lazy was not as far as I recall.

Yeah, I played Fortnite a lot, and one day decided to get a skin. I didn't realize they were randomized, so I ended up hunting skins for hours until I managed to hunt something down. Then it hit me: what the hell just happened to me? It was like those ads that beg you to click them to see the mystery, and when you do the mystery is just hidden behind more clicks.

I deleted my Fortnite account and never looked back (and never even used the skin I bought). I know addictive behavior in myself when I see it, and that wasn't going to end well.

Did you actually read the essay?

As one person who likes thinking to another, I think you'd enjoy it. And after you read it, you would have much more specific things to insult me over. In my experience, precise, targeted, and entirely true insults are far more gratifying.

WELL WELL WELL. We meet again, my favorite CS department member. Alright here is an essay for you: It suggests o3 is qualitatively different than older generation LLMs. I look forward to your take on it.

https://ancillary-proxy.atarimworker.io?url=https%3A%2F%2Farcprize.org%2Fblog%2Foai-...

I was more or less thinking that it would be untraceable in the context of launching a second strike in response to a decapitation strike. In theory, it might eventually get back to the country that launched the strike. In practice, it would take a very long time in terms of mounting an effective second strike. I guess nuclear subs could just hang out until it was established who-dunnit?

Well yes. But nukes that can't be traced back to their origin is definitely the plot to a video game.

METAL DRONE!

Yup. A large drone could easily carry a hydrogen bomb. And we wouldn't know who launched it, and thus no retaliatory response would be possible.

I was giving a description of the sort of object an LLM is, not giving advice on how to "ride one" so to speak.

My main points are:

1. It was intended to mimick the way certain signals are amplified and others downregulated as information propagates along a neural network in animals.

2. It's a profoundly complex system that is not well understood.

As a result, I am skeptical since I've yet to see any substantial explanation for why it's just "next token prediction" practical or otherwise. The explanations I've seen focus mostly on the objective function, which doesn't really reflect the intrinsic nature of the LLM as much as it is just a way to measure how the parameter variation is doing. If you have a good way to break this down or explain it, practical or otherwise, I'm all ears.

I've noticed at least two universities in which the relations between the math department and CS department were icy. I'm starting to see why.

First, I need to clarify something. Fractals can come about in a number of contexts and situations, and how they get defined can vary a bit. The element of "fractalness" that they strike me as having comes really from fractals that involve feeding an output in as an input into some fixed function, repeatedly. When that is done over the whole domain of a function, sometimes something fractal will pop out. In the case of something like a Julia set, I can feed the output of a function back as an input into a quadratic function over the complex numbers, and eventually it may or may not escape to infinity. If this is done to enough points, some fairly intricate behavior can show up. Incidentally, there are some fields - such as arithmetic or algebraic dynamics - where the whole idea is to see what happens when you carry out that process of feeding in outputs as inputs. Incidentally, that's fairly similar to what goes on with Julia sets.

Now, I may be mistaken about the newer LLMs, they could be far different from networks I studied. But at least with a deep network, you basically have a sequence of linear transformations (represented by matrices) along with non-linear "activation" functions in between those. So we have "matrix + non-linear function," and then output of that gets fed into "matrix + non-linear function." Notice that's fairly similar to what happens in some dynamical systems, where an output is fed back in as an input. In fact sometimes the data gets fed back in to previous layers of a network full-stop. So strictly speaking, that's not the same as arithmetic or algebraic dynamics, where a polynomial might have an output fed back in as an input, since in this case the "polynomial" gets changed at each step. Still, at a most fundamental level, at least many modern networks operate by "feeding the output into a function as an input," even though for networks the function itself can be tampered with between iterations.

Now, as for "fractal like," notice the outputs of neural networks will be somewhat similar as various inputs change. For example, as you change an input string, the output string will change in ways that reflect the semantic change of the underlying sentence. Sometimes there are dramatic changes, sometimes not. That is very much fractal in nature, where recurrent patterns appear as we move around the domain. I'm fairly confident that if we did it carefully, such as measuring various facets of the input-output behaviors to plot them, we could probably get some very dazzling fractal type behaviors.

The structure of an objective function for a neural network is horrendously complex, so really it should have a lot of local minima. Yet when they're trained, the algorithms still manage to find fairly nice almost global minima, despite the fact there should be a lot of local minima hanging out to cause problems. That is what's weird that the training algorithms work. If we knew the space was basically a big paraboloid in higher dimensions sure. But really, given how an objective function works, that really shouldn't be the case, and those local minimas should cause more problems than they do.

Human students do that too, it's called top-down processing, which can be illustrated by the "your brain can do amazing things" activity with the middle letters of the words all mixed up, but we can still read it. Abuses of top down processing also show up in jokes, like "How many animals did Moses bring onto the ark?" followed by "Ha! Moses didn't bring animals onto the ark, that was Noah!" When I teach a class, I am always vigilant for places a student might use top down processing to botch something and give warning, because that is very human. Basically, I don't think an LLM making a mistake humans commonly make is a great litmus test. Try giving it a warning to be "careful with the steps" and see what happens.

I'm not really sure about the buzzy "it's just next word prediction" chatter. It's more of a highly non linear dynamical system that have parameters tuned until an objective function is minimized. In this case, the objective function measures - along with over fitting penalties - how well the system outputs resemble spoken language. If we scrambled a person's brain, as their brain healed their "output" would also start to minimize the objective function, so to speak. Basically, we're messing with a very complex non linear fractal dynamical system that was modeled on organic nervous systems, until it starts to respond in a way statistically similar to a person. I'm not comfortable with just handwaving that as "just next word prediction."

I mean, it's even weird stochastic gradient descent even works. The associated graph should really have a lot of local minimums to throw things off, but for whatever reason, it still manages to train. Something very strange happens up in hugely dimensional space, and it allows the training to work even though it really probably shouldn't.

o1 is "deactivated" as soon as it gives its response. It's one input in, one response out. The only way it could attempt ex-filtration is if this is an agent version of o1.

No, it isn't just that. The bill of rights also outlines our rights, which are mentioned in the declaration of independence: "That to secure these rights, governments are instituted among men." It does not say "governments instituted shall not infringe upon these rights." The entire purpose of government is to secure our rights. The fact the first amendment outlines that freedom of speech is a right, means that the entire purpose of government is to secure the right to free speech, among other rights.

Personally I'm on the verge of quitting The Verge, too. Yesterday it irked me how they intentionally buried the o1 announcement and headlined Genie 2. Despite the not so subtle attempts to play whatever game that was, the o1 coverage still ended up #3 in the most popular list. I don't even feel like unpacking all of the ulterior motives and back room dealings that likely involved. It reminded me a bit of that time the media collectively decided to ignore Trump, hoping that would make him go away. Nope. Object permanence is definitely still a thing guys, nice try though.