I happen to have close personal friends both in film making, and manufacturing.

I think this back-and-forth reveals just why these tariffs don't make any sense, and very likely won't lead to any meaningful outcomes.

It's less about the validity of the arguments, and more about this exact conversation/debate playing out thousands of times with hundreds of lawyers, accountants, producers, executives, government officials, etc. There's so much ambiguity in Trump's executive order that it's very close to meaningless to use as a template to plan around. He may as well have ordered all actors wear "turquoise", and everyone would be left arguing over what shade he actually meant.

This is a repeating pattern with Trump. Almost everything that comes out of his mouth is a verbal equivalent of a Rorschach Test --he makes a bunch of ambiguous, non-specific statements/declarations/promises and everyone listening fills in the blanks to create their own image of what they hope it actually means.

This back-and-forth is yet another example of this. He's got you two arguing over what his verbal ink blots actually look like.

Trump put zero thought into this latest verbal ink blot, which means you two shouldn't either.

The Kremlin cut flows of Nordstream 1 down to 20% of max capacity, and then completely under the guise of "maintenance". They would restart, and shut down, transport sporadically after that (also under the guise of maintenance) at the time Germany and the U.S. were bouncing the idea of sending tanks to Ukraine.

The order of events wasn't:
NG is cheap -> Kremlin shuts gas -> NG remains cheap -> pipes destroyed -> NG is expensive.

It was:
NG is cheap -> Kremlin shut gas -> NG is expensive -> pipes destroyed -> NG remains expensive.

Expensive NG was not due to pipe destruction. It was the Kremlin.

And, the Nordstream 2 pipeline wasn't even in use when it was destroyed.

So no. Gas prices did not go up because somebody blew up an unused nat gas pipe.

The higher prices were due to the Kremlin reducing gas exports to Europe in an effort to drive prices higher and pressure European governments to stop supplying Ukraine with lethal aid.

Once the Kremlin raised prices by reducing flows, some unnamed government realized the following scenarios:
A) High gas prices, Russia has leverage over Europe
B) Destroy pipeline, high gas prices, Russia loses leverage over Europe

High prices were not due to the destruction of the pipeline.
The high prices were due to the Kremlin.

"But what you don't need is a term for the population of earth minutes 10k or so with that mental illness."

Says who?

The Inuit in Canada has over 50 words for snow. If the rest of the world has no need for that many words for snow, who are we to deny/prevent the use of those 50 words?

Same argument can be made against the naming of the wavelengths of light that are subjectively referred to as "blue".
"But what you don't need is a term for the colours minus the 10k or so with that shade or hue". --Yeah, it's called "not blue".

Oh wow. He still has the page digit dreaming page (you need Flash Player)
http://www.cs.toronto.edu/~hin...

Main page is here:
http://www.cs.toronto.edu/~hin...

Back when I was learning about stacked Restricted Boltzmann Machines, you could visualize the network's "hallucinations" by introducing noise to the latent space (hidden layer), before recreating the input on the backward pass (to the visible layer). You could take this "corrupted" recreation of the input, pass it back into latent space, introduce noise, and then pass it back to the visible layer to see a new hallucination.

Hinton had a nice animation of this on his U of T website back in the day. You could select a digit (like, the number 5), and then just watch his stacked RBMs hallucinate wobbly, morphing, squished, stretched, distorted 5's in real-time. At the time, it definitely looked to me I was watching a machine "dream".

So, let's suppose this actually was what dreaming is. Why is it useful? Why does practically every living creature with legs and a face dream? How does being incapacitated, inattentive, and vulnerable for 1/3rd of the day to dream lead to improved odds of survival?

Dreaming floods the brain with "fake training data". Like today's AI companies, large complex neural network are short on data. And the same is true for human brains as well. Upon receiving new impactful data, the human brain creates fake data to use as training material. How can fake data help the brain? Hinton figured that in training RBMs, he could get his networks to "learn quicker" if he (I'm simplifying a little here) let his network "dream a little bit" before calculating weight updates (for those who interested, he introduced several rounds of gibbs sampling). Allowing the network to "dream" and create wacky "fake" data made the weight updates much more impactful because instead of teaching *what something was*, it taught the network *what it wasn't*. Dreaming allowed the network to diverge from reality, and when the weight updates were literally "reality minus imagination", the bigger the imagination, the bigger the weight updates.

To understand this in layman's terms, imagine you've never seen a teapot before. When you dream about a teapot, your brain is teaching itself all sorts of wacky teapots. Large teapots, tall teapots, teapots shaped like animals, and teapots that can fly. When you wake up, reality creates a massive weight-update to your brain's connections about what a teapot actually is. A "rude awakening". I strongly suspect this is why sleep has evolved to be necessary. There's not enough "real" training data in reality for brains to learn what "stuff is", so we dream to learn what it's not.

I suspect the challenges of using output-as-input can be solved somewhat by implementing something resembling sleep.
Of course, we living creatures have reality (hopefully). Not really sure what "reality" means for large nets though.

https://ancillary-proxy.atarimworker.io?url=https%3A%2F%2Farchive.is%2FqgBWB

Outcomes like this exposes what will make "AI" truly revolutionary. Many people are trying to tackle this "garbage-in-garbage-out" problem by shifting focus on training on more "authoritative" sources of data, like textbooks and the like. The concern here is deciding what is "authoritative" when it comes to determining "the truth".

I believe this is the wrong approach. "Truth" is not determined by the authoritativeness of its source. The "truth" is "the truth" because it is consistent with many other observations and phenomenon. An extreme example would be like a detective determining the truth despite being fed nothing but lies. Here, the "truth" is determined or discovered through the application of deduction and reasoning. This is what's missing from "AI" today --logic and reasoning.

The latent states of today's AI models don't cycle through a logic engine. Large GPT models are simply very large feed-forward models that has layers slapped on to make them taller to compensate for the fact that it can't "think about" its "own thoughts". A 50-layer model basically has a limit of 50-cycles of "thinking" to come up with a grammatically-correct and contextual response. Add this to the fact that future words in a potential response are heavily dependent on words generated earlier. In simple terms, these models are "pot committed" to the words it has already generated. Combine these two factors together, and you end up with models that essentially "speak without thinking much", and what they *do* say resemble modified regurgitations of what it has already seen.

I suspect the solution to this will be a change in network architecture that allows for latent states to propagate cyclically, and concatenated with latent activations that represent "questions" or "challenges" without limit, until the resulting activations reach steady-state.

TLDR; I believe these "garbage-in-garbage-out" issues with ChatGPT-based LLMs can be resolved if these models are able to talk/impose-challenges to itself for some time before it generates the first word of its response to the user.

But that's just my opinion.

Yeah I see where you're coming from. The problem(?)/non-problem(?) with logic is that it relies on starting assumptions. Newtonian physics works wonders until somebody imagines what it's like to travel at the speed of light. The universe revolves around the earth until somebody imagines what would happen if everything revolved around the sun. Roots of negative numbers are ignored, until somebody imagines what it'd be like to simply leave it in an equation.

I don't see hallucinations as a bad thing. What I see now are LLMs being trained with inordinate quantities of noise, simply because it's the simplest thing to do to avoid overfitting. It's definitely easier than dreaming up a revolutionary network architecture like the Transformer. So I see a future where LLMs are smaller, more specialized, require far less data to train as a result, and their levels of imagination are adjusted during training to fit the environment/task they're designed for. Today, everyone's going for size, size, and more size, simply because it's the simplest thing to do. That is, until somebody realizes there's not enough information on the internet to train with, and is forced to rethink their goals.

The real test of a truly useful reasoning AI is if model trained on highschool physics can reason with itself to deduce special relativity. When that happens, there will be another race in AI; one that matches the excitement that followed ChatGPT. Such a model will not be as large as ChatGPT, and require far less data. I say this with confidence because Albert Einstein did not require terabytes of the world's information to deduce special relativity. The level of math is within the reach of many 18 year-olds. The difference between discovering special relativity, and not, is imagination.

The designers of today's LLMs have not decided if they want their models to excel at information recall (overfitting and information indexing. ie. "Google"), or "understanding". Understanding and reasoning is a hard problem to solve, but I think we'll get there. My other comment on this slashdot post details why recall and understanding may be mutually exclusive goals. It's the classic battle between "bias" vs "variance".

I'm not disagreeing with you, but the source of the hallucinations is a little more nuanced.

When training a neural model, one of the goals is to avoid "overfitting" (or "memorization"). You can think of it as, somebody thinking cats can only be black, because all the cats that they've ever seen were black. So in their mind, it's impossible for cats to be any other colour. To avoid this scenario, models would need a shit-tonne of data (as many pictures as possible of all the world's cats), or.... "noise" can be added to a "higher layer" of the model, where "colour" is understood. This "noise" could change the colour to blue, green, pink, etc, at random. The upside is overfitting will be less likely (which is good), the downside is you'll end up with a model that thinks purple and green cats are just as "real" or "common" as white cats. So now when somebody queries this model with "What colours can cats be?", it could respond with colours like purple and green.

This is a generalization, but it illustrates the source of "hallucinations".

Sidenote: This can also help with understanding why LSD results in hallucinations, and the mechanism by which it improves neural disorders such as PTSD and addiction. PTSD and addiction can be the result of strong neural connections/associations the brain creates during extremely emotional and traumatic events. This "over-weighting" is how the brain "overfits". LSD may chemically weaken the strength of these overfit neural connections and allow other signals (noise) to propagate where they previously couldn't --leading to hallucinations.

Back on topic.
Hallucinations by themselves are not *all* bad. They're just uncontrolled imagination. The question is, how much imagination is appropriate/desired/acceptable in a given context? eg. Image generation, information recall, image comprehension, legal document paraphrasing, grammar-checking, programming code analysis/debugging, etc. These would all have different levels of imagination that we would deem as acceptable.

I think AI companies are misguided in their efforts for two reasons.

First, the internet being "too small" is a consequence of attempting to create a "jack of all trades" model. It would be equivalent of trying to train a single human to be knowledgeable and experienced in every single academic subject known to man. Now, "so what", right? They can burn all the money trying for all we care. My point here is if these companies want to produce anything useful that can be sold, and they care about efficiency, then the approach to take would be to create many smaller specialized models, instead of one gigantic that'll be useful to everybody for anything and everything.

Second, these companies need to decide for themselves if they want an AI to be great at recalling information, or great at "understanding" information. ie. A Google-style model to be used for information recollection will under-perform at "understanding" information. This is the classic bias-variance trade-off. A human example would be the person the movie Rain Man was based off of (Kim Peek). He was a person who had almost perfect information recall, but when neuroscientist Vilayanur Ramachandran performed a few scans, he noted that Peek's corpus callosum was missing entirely.

The corpus callosum connects the two halves of the brain, and in a way functions as the narrow "choke point" of an autoencoder, that forces similar information close to each other in "understanding space". In Peek's case, he was able recall every single random fact asked of him, but when he was told "get a hold of yourself", he would literally grab one of his own shoulders with the opposite hand.

These companies chasing data mindlessly are doing so because they simply do not understand the end goal of their efforts. Do they want great recall? Great, hunt down as much information as possible. But what they'll end up with is a data indexing system that is one generation more advanced than Google's, and nothing more.
Do they want great understanding? ie. A model that can relate facts, reason, and "understand" what people "mean"? Then they'll have to abandon the obsession with data and focus on developing the architecture of the model itself and train it with a specialized subset of all the world's data.

It's possible for there to be a middle-ground somewhere, but I imagine the resources needed to acquire the data and train to be enormous, and it may only be practical only if the goal is to out-Google Google.

A lot of people have kids because they want kids, but don't want to be parents.

I imagine AI replacing management would be extremely successful as well. The information inputs that feed into management decisions are considerably less noisy than the inputs that "ground level" employees interact with on a day-to-day basis. And, there's far less variation in not only the types of scenarios that management faces, but also in the decisions they can possibly make. eg. Ground level data is aggregated, distilled, and compressed into easily-understood metrics; and the basis that forms the backbone of management decisions should already exist in business management and project management textbooks.

[Correction] Kids *don't* have...