I know this is an alien concept to most people here, but it would be nice if people would actually, you know, read the papers first? I know nobody does this, but, could people at least try?

First off, this isn't peer reviewed. So it's not "actual, careful research", it's "not yet analyzed to determine whether it's decent research".

Secondly, despite what they call it, they're not dealing with LLMs at all. They're dealing with Transformers, but in no way does it have anything to do with "language", unless you think language is repeated mathematical transforms on random letters.

It also has nothing to do with "large". Their model that most of the paper is based on is minuscule, with 4 layers, 32 hidden dimensions, and 4 attention heads. A typical large frontier LLM has maybe 128 layers, >10k hidden dimensions, and upwards of 100 or so attention heads.

So right off the bat, this has nothing to do with "large language models". It is a test on a toy version of the underlying tech.

Let us continue: "During the inference time, we set the temperature to 1e-5." This is a bizarrely low temperature for a LLM. Might as well set it to zero. I wonder if they have a justification for this? I don't see it in the paper. Temperatures this low tend to show no creativity and get stuck in loops, at least with "normal" LLMs.

They train it with 456976 samples, which is.... not a lot. Memorization is learned quickly in LLMs, while generalization is learned very slowly (see e.g. papers on "grokking").

Now here's what they're actually doing. They have two types of symbol transformations: rotation (for example, ROT("APPLE", 1) = "BQQMF") and cyclic shifts (for example, CYC("APPLE", 1) = "EAPPL".

For the in-domain tests, they'll say train on ROT, and test with ROT. It scores 100% on these. It scores near-zero on the others:

Composition (CMP): They train on a mix of two-step tasks: ROT followed by ROT; ROT followed by CYC; and CYC followed by ROT. They then test with CYC followed by CYC. They believe that the model should have figured out what CYC is doing on its own and be able to apply CYC twice on its own.

Partial Out-of-Distribution (POOD): They train on simply ROT followed by ROT. They then task it to perform ROT followed by CYC. To repeat: it was never traiend to do CYC.

Out-of-Distribution (OOD): They train simply on ROT followed by ROT They then task it to do CYC followed by CYC. Once again, it was never trained to do CYC.

The latter two seem like grossly unfair tests. Basically, they want this tiny toy model with a "brain" smaller than a dust mite's to zero-shot an example it's had no training on just by seeing one example in its prompt. That's just not going to happen, and it's stupid to think it's going to happen.

Re, their CMP example: the easiest way for the (minuscule) model to learn it isn't to try to deduce what ROT and CYC mean individually; it's to learn what ROT-ROT does, what ROT-CYC does, and what CYC-ROT does. It doesn't have the "brainpower", not was it trained to, to "mull over" these problems (nor does it have any preexisting knowledge about what a "rotation" or a "cycle" is); it's just learning: problem 1 takes 2 parameters and I need to do an an offset based on the sum of these two parameters. Problem 2... etc.

The paper draws way too strong of conclusions from its premise. They do zero attempt to actually insert any probes in their model to see what their model is actually doing (ala Anthropic). And it's a Karen's Rule violation (making strong assertions about model performance vs. humans without actually running any human controls).

The ability to zero-shot is not some innate behavior; it is a learned behavior. Actual LLMs can readily zero-shot these problems. And by contrast, a human baby who has never been exposed to concepts like cyclic or rotational transformation of symbols could not. One of the classic hard problems is how to communicate with an alien intellect - if we got a message from aliens, how could we understand it? If we wanted to send one to them, how could we get them to understand it? Zero-shotting communicative intent requires a common frame of reference to build off of.

100% this. I'm a vegetarian, the sort of person they think should be buying their products, but their products disgust me, because they remind me of meat. I don't want to be reminded of an animal corpse while I'm trying to enjoy a tasty meal. Why mimic the thing I don't want to eat?

(I'll only speak re: vegetarians below, but I expect vegans are similar)

I would ask non-vegetarians: imagine that you live in a world where people ate toddler meat. Real human toddlers, slaughtered for their meat. The vast majority of people in your situation would of course avoid eating them. Some may be radical anti-toddler-meat campaigners. Others may silently accept that they're not going to change the rest of the world. Either way, you let people know that you don't eat toddler meat so they don't serve it to you. But hey, some friendly cooks feel bad that you don't get to enjoy toddler meat! So they make a baby meat substitute that looks and tastes exactly like toddler meat! They package it in packages with pictures of dead toddlers on it, but with labels "No toddler included!" And then they expectantly wait for you to thank them and praise them for finally making toddler meat that you can eat - rather than being disgusted by the whole concept and wanting some non-baby related food that doesn't make you think about dead toddlers while you eat.

That's not the situation *all* vegetarians are in, but it is the situation that a *lot*, dare I say most, vegetarians are in.

I think a lot of non-vegetarians cooking for vegetarians are just frankly confused about what to offer us, as they have trouble picturing of a meal without meat. It's really simple: you know how fatty, salty, carby, umami-rich stuff tastes really really good and leaves you feeling satiated? Yeah, just make something that's fatty, salty, carby, and umami-rich that doesn't involve meat, and your vegetarian friends will be happy ;) Like, pick anything carby, cook it with a tasty fat that pairs well with it, add salt and something umami-rich (things like mushrooms, nuts, tomatoes, yeast, nori, olives, nuts, cheese (vegetarians only), spices, etc etc), and viola, that's a good vegetarian dish ;)

  Of course, *to make it healthier*, and a more "adult" taste, you'll want to include non-carby veggies (which, per unit *dry mass*, are actually highly protein rich, with e.g. freeze-dried broccoli being well more protein rich than your average grade of ground beef without its water, and freeze-dried watercress being up there with fish - they're just heavily watered down). Veggies also add umami. You can also - optionally, but it's not at all a requirement - include high-protein things like tofu, tempeh, seitan, TVP, etc. But protein deficiency is not common among vegetarians or vegans in western society (the main risk is iron deficiency, particularly for vegans, and - exclusively for vegans - B12 deficiency, but only if they don't eat anything fortified with B12, though B12 fortification is common).

Am I the only person getting whiplash that we're rediscussing the exact same thing when this concept was already proposed as Breakthrough Starshot, and was big in the press at the time, incl. on this site?

Anyway, you still have to have the energy to transmit back, which was proposed to be from a RTG: My hot take: since you already have to have a (comparably) big sail anyway, which means booms to maintain its structure, use 232U to get many times the energy per gram as 238Pu (38,9 MeV vs. 5,6 MeV), at the cost of a hard gamma from 208Tl, and put it out on the ends of the booms, with appropriately radiation-hardened electronics. You could also do double-duty with an alpha sail (alpha emitter backed by a thin low-Z material to create net thrust from alpha emission)

Also, if the 232U is in the form of a compound that's sufficiently soft for fast diffusion at its equilibrium temperature, you can diffuse out the 220Rn and avoid 208Tl's hard gamma altogether. This costs you (you only capture 16,6 MeV of alphas), but not only does it avoid the hard gamma, but it also means that you don't retain the mass of the stable decay products, so your craft gets significantly lighter over time. (At one point I was considering urania aerogels to lose the radon instead of "soft" high-diffusion materials, but the data was suggesting that the aerogel would quite quickly self-pulverize and densify)

232U is readily producible (indeed, it's a waste product in thorium reactors); main issue is just that it's a pain to handle. But for something like this, you're dealing with microscopic amounts.

Lithium aspartate, for example

The least-harm principle. There's essentially universal agreement that low (dietary-range) levels of lithium are not harmful, while the research as a whole is strongly suggestive of a benefit (but has not yet met the standards of, for example. an EPA regulatory standard for lithium in drinking water). Lithium, at the doses necessary, costs basically nothing, takes seconds to take, and is orders of magnitude away from the levels where potential toxicity symptoms can arise. To me, that's an easy call. Also, Alzheimer's runs in my family, so there's an extra factor weighing on the scale.

No issues with aspartate. It dissociates fully upon dissolution. L-aspartate is a building block to proteins. D-aspartate is also naturally found and used in the body.

All of the salts you listed fully dissociate in water.

Nothing weird about sodium fluoride, fluorosilicic acid, or sodium fluorosilicate. Sodium fluoride is a simple salt, dissociates immediately upon dissolution to Na+ and F-. Fluorosilicic acid and sodium fluorisilicate result in a fluorosilicate ion (SiF-2) which rapidly hydrolyzes to Si(OH)4 + 6F- + 4H+. Si(OH)4 (orthosilicic acid) is the form of soluble silicon which plants and diatoms consume and is perfectly normal in water in the tiny amounts from fluoridation (like 6 micromolar concentration). Ocean surface water near Antarctica for example is up to ~80 micromolar concentration. And it goes without saying that minuscule amounts of sodium in water are also perfectly normal. The addition of the fluoride ion is the only actually meaningful impact.

Dial "996" for more information.

Really annoying that lithium orotate became the standard way to deliver lithium, given that orotic acid is toxic and mutagenic :P

TL/DR, it affects the enzymes that regulate these processes.

Lithium is naturally present in the diet, but it varies by orders of magnitude depending on where you get your water and where your food was grown / grazed, with most people today on the lower end of the intake. Mineral spring waters in particular tend to be much richer in lithium than river / lake water, and also the fact that municipal water supplies' range limitations on the quantities of common minerals (sodium, potassium, calcium, etc) will also tend to reduce lithium, could be argued that, on average, the average person in the past might have consumed more. But it still would be quite varied on a regional basis.

Note that drinking lithiated water used to be a popular health trend. Indeed, 7-Up was originally called 7up Lithiated Lemon Soda (though the claim of being lithiated was actually a lie in their case, and they ultimately had to remove it!).

It always strikes me as weird that people would be shocked that a substance commonly prescribed to affect the brain... affects the brain.

Beyond increasing BDNF activity (which promotes neuron survival and new growth), one of the main therapeutic targets is Glycogen synthase kinase-3 beta, which controls a wide range of developmental parameters for neuronal development, including discouraging regeneration and promoting apoptosis, and is pro-inflammatory (CNS inflammation is itself associated with Alzheimers). Lithium reduces its activity, both with direct and indirect inhibition. While GSK-3B is essential to a degree, overactivity of GSK-3B is associated with a wide range of neurodegenerative diseases, including Alzheimer's.

Lithium also helps promote cleanup (misfolded proteins, degradation products, etc) via autophagy by reducing the activity of IMPase... at least at low doses. At high doses, it can cause the inverse effect, due to its GSK-3B and mTOR impacts. In general, though, it seems to typically be pro-autophagy.

There have been quite a few studies that do just that. Well, not "taking lithium", as in the medication, because typical psychiatric doses of lithium (hundreds of milligrams per day) are like 2-3 orders of magnitude higher than typical dietary doses. But dietary doses themselves vary by orders of magnitude (some European studies put consumption in some places in Europe as low as a couple micrograms per day, while in places in northern Chile some people consume ~10mg per day), because lithium is so widely varied from place to place. One study found for example that one Slovakian bottled water product had 10 milligrams per litre, while the mean European bottled water had less than a microgram per litre. In most places, peoples' dietary consumption is closer to the lower end than the upper end. And the studies strongly suggest that people who naturally consume the higher amounts of lithium have lower rates dementia (there's also positive, though weaker, evidence for lower rates of violent crime and suicide). In general, it seems to be neuroprotective.

It's been argued that we should probably be lithiating water, e.g. that there should be minimum and maximum standards for lithium in drinking water the same way that there are minimums and maximums for numerous other minerals, with a provisional recommended daily intake of 1mg per day based on the evidence. But given the huge backlash to fluoride in water, I can't even imagine how harsh the backlash to lithium would be, given that people associate it with being a psychiatric medication (even though that's at doses orders of magnitude higher). It's just not going to happen.

I personally take 1mg of lithium a day. Which is well within the normal dietary range (in some places in northern Chile people naturally consume ~10mg per day!). A common supplement form is lithium orotate, but it's a weird choice - it's chosen because it's covalently bonded into a molecule which is delivered into cells whole, to "make it more effective", but A) that's not how normal dietary lithium is delivered, and B) orotic acid isn't exactly healthy. Instead, I make my own (both concentrate and diluted solution). I start with lithium carbonate, and while it's not available in food grade (anywhere I've found), it's a very common compound available at high purity (>99,5%), with easy composition tests - crimson flame test, density tests (offset by a typically poor packing density), low solubility in water but high bubbling solubility in weak acids (with no precipitate), etc). Because it's poorly soluble & tastes like baking soda / mineral water, I also add citric acid to the solution, forming lithium citrate. Even if the impurities were pure lead, the amount would still be small when you're only taking 1mg a day. But actual impurities are mostly (A) water, (B) other lithium compounds (hydroxide, chloride, sulfate, etc etc), and (C) other similar mono- and divalent cations to lithium, such as sodium, potassium, calcium, magnesium, plus some iron, alumium and silicon due to their ubiquity in nature and presence in processing.

china doesnt let you exploit workers

Oh my God, I had to stand up and walk around after reading that to catch my breath, as I was laughing so hard.