It is interesting that they manage to play a game to the end, but there is no point to have them play in a competition.
Oh, definitely. I wasn't musing this in the sense "let's send a chatGPT-powered chess engine to a tournament !",
more like "maybe a chatGPT-power chess engine could manage to play more than a couple of round, enough to keep your nephew entertained".
It suddenly occured to me to ask a side question of whether it could recognize this character from these contour points, and it could. It said it "looked" like the letter 'a'.
You only asked it once. It could be any of:
- You got one lucky answer. I could be that chat bot are bad at classifying glyphs, you just got one lucky answer (see: spelling questions. Until byte-based input handler are a thing, a chatbot doesn't see the ASCII chars forming a string, it only sees tokens -- very vaguely, but good enough metaphor: words) (Same category: all the usual "Chat bot successfully passes bar/med school/Poudlard final 's exams better than average students" press releases). But give it any other glyph of your current work and the overall success rate won't be higher than random.
- It actually answers "a" to everything. Give it any ROT-13 encrypted text, the older chatGPT on which we tested that always answers "Attack at dawn" (see: Cryptonomicon)
- LLM could be not too bad classifiers. It's possible that (having trained on everything that was scrapable off the internet) the LLM's model has learned to clasify at least some gylphs better than random. (Again you'd be surprised at what was achieved in handwriting recognition with mere HMMs) (And LLMs have been applied to tons of other pattern-recognition task that aren't really human language: it's been used in bioinformatics for processing gene and protein sequences)
Just as an exercise: take a completely different non-letter vector object encdoded the same way. Replace the vector in your question with the bogus vector, and keep the rest of the question as-is, including any formulation that would be putting the bot on a certain track (e.g.: if the original question was "what letter of the alphabet is encoded in the following glyph" keep that part as-is). And ask it to explain what it saw using the exact same question.
Repeat with multiple fonts but other letters, and multiple non-letter objects.
Does it consistently answer better than random? Somehow recognise the non-letters (calling them emojis or symbols if the prompt forced it to name a letter)?
Or does it call everything "a"? Or does it only successfully recognises "a"s and "o"s but utterly fails to recognize "g"s or "h"s ?
How in the world can a language model do that?
Again. HHMs, LLMs trainged on bioinformatics sequences.
The data had been normalized in an unusual way and the font was very rare. There is zero chance it had memorized this data from the training set.
"rare" and "unusual" don't necessarily means the same thing to human eyes and to a mathematical model.
It doesn't need to literaly find the exactt same string in the training set (it's not C/C++' strstr() ), it merely needs to have seen enough data to learn some typical properties.
And if you look at how some very low power retro tech used to work for handwriting recognition: Palm's Graffiti 1 didn't even rely on any form of machine learning. Just a few very simple heuristics like total length travelled in each cardinal direction, relative position of the start and stop quadran, etc.
So property could be "the vector description is very long" which well within what a typical language model could encode.
And again that's assuming that the chat bot consistently recognises glyphs better than random.
It absolutely must have visualized the points somehow in its "latent space" and observed that it looked like an 'a'.
Stop anthropomorphising AI, they don't like it.
Jokes aside: LLMs wont process any thing visually. Just most like words given a context and their model. Yes a very large model could encode relatioship between words that corresponds to visual propertise. And it is plausible that given enough scraped stuff from the whole internet, it has learn a couple of properties of what makes an "a".
Buit yet again that's assuming that the chat bot consistently recognises glyphs better than random.
I asked it how it knew it was the letter 'a', and it correctly described the shape of the two curves (hole in the middle, upward and downward part on right side) and its reasoning process.
It's an LLM. It's not "describing what it's seeing". It's giving the most likely answer to what an "a" looks like based on all it has learned from the internet.
Always keep in mind that what a chatbot gives you isn't "What is the answer to my question?", but it gives "How would an answer to this question convicingly look like?".
There is absolutely more going on in these Transformer-based neural networks than people understand.
Yes, that a totally agree. An oftern completely underlooked aspect is the interpretation that goes in the mind of the homo sapiens reading the bot's answers.
I could joke about seeing Jesus in toats, but the truth is that we are social animals, we are hardwired to assume there's a mind whenever we see a realistic and convinving language. Even if that language is "merely" the output of a large number of dice rolls and a "possible outcomes look-up table" with a size thats incomprehensible to the human mind.
It appears that they have deduced how to think in a human-like way and at human levels of abstraction, from reading millions of books.
"appears" is the operative key word here. It's designed to give realistic sounding answers.
Always. It always answers, and it always sounds convincing by design, no matter how unhinged the question actually is.
The explanation looks "human-like" because the text-generating model has been training on a bazillion of human-generated texts.
In particular they have learned how to visualize like we do from reading natural language text.
Nope. Not like we do, at all.
But they are good at generating the text that makes it look like that it would be like we do.
Because again, they are good at language and that what they are designed to do.
At absolute best situation, one of the latest generation multimodal models, that not only do text but also is designed to do process image (the kind of chatbot to which you can upload image, or which you can ask to generate image), could be generating some visuals from the prompt and then trying to text-recognition on that intermediate output.
It wouldn't surprise me at all if they can visualize a chess board position from a sequence of moves.
Given a large enough context window, it could somehow keep track of piece positions.
But what pro players seem to report is that current chatbot actually suck at that.
nah, this will soon come crashing down as the enshitification of commercial games continues.
Valve itself is NOT publicly traded. There are no shareholders to whom the value needs to be shifted.
This explains (in parts) why Valve has been a little bit less shitty than most other companies.
It also means Valve's own product (Steam, SteamDeck, upcoming Deckard, etc.) are slightly less likely to be enshitified
(e.g.: whereas most corporations try to shove AI in any of their product, the only news you'll see regarding Vavle and AI is Valve making it mandatory to label games that uses AI-generated assets)
if i really want a game i wait until the price seems reasonable and affordable even if that means waiting for years, the side benefits are there's more content, most of the bugs are squashed and the drama is history, it seems unethical to support classist corporations in any fashion especially financially in my view
Also indie games are a thing.
Indie-centric platform like itch.io are a thing.
Unlike Sony and Microsoft, Valve isn't selling the Steamdeck at a loss, so they care less where you buy your games from -- hence the support for non-Steam software (the onboarding even includes fetching a browser flatpak from FlatHub).
Humble bundles are also a thing (with donation to charities in addition to lower prices).
So there are ways beyond "buy a rushed-to-market 'quadruple A' game designed-by-comitee at some faceless megacorp".
It's expected. At their core all chess algo are search a min-max tree, but instead of going width- or depth- first exhaustive search, they use heuristics to prioritize some branches of the tree (A-star).
On the modest hardware of the older machine there isn't that much you can explore before the player gets bored waiting.
So obviously, you're going to make much stringent rules: "Never take a branch where you lose a piece" prunes entire swaths of the tree, rather than "see if sacrificing peice XXX gives us a better path" which would require exploring more of the tree.
Having been trained on all the corporation could scrape from the internet, I would expect an LLM to have been trained from a lot of real-world chess games (e.g.: reports of games from online archives of chess magazines, etc.), so as long as the tokeniser is able to parse the notation found there (or at least distinguish the moves. It doesn't really need to be able to "understand" the notation into english), it has a large corpus of "lists of moves leading to a win" which would include real moves (sacrifices as you mention).
And given a large-enough model to encode that, would be in the same ballpark of the hidden Markov models of yore -- provided it keeps track of the current state of the game (which it currently does not).
I wonder if you wouldn't win if you just told ChatGPT to write an chess AI and then used the chess AI to beat the Atari. Writing code is something text models are good for. Playing chess is not.
The devil is in the detail.
All chess algorithms are A-star: they search a min-max tree, but use heuristic to prioritize some branches instead of doing width- or depth- frist.
Generatingn a template of a standard chess algo would be probably easy for a chatbot (these are prominently featured in tons of "introduction to machine learning" courses that training the LLM could have ingested), writing the heurisitc function to guide the A-star search is more an art-form and is probably where the chat bot is going to derail.
Funnily though, I would expect that if you used the chatbot AS the heuristic it wouldn't be a super bad player.
Have some classic chess software that keep tracs of the board and lists all the possible legal moves, then prompt the chatbot with something like:
"This is the current chessboard: {state}, these at the last few moves: {history}, pick the most promising among the following: {list of legal move}".
In fact, decades ago that's how some people have applied hidden Markov models to chess.
Similarly, I would expect that during training, the LLM would have been exposed to a large amount of all games available only, and has some vague idea of what a "winning move" looks like given a current context.
Not much trying to simulate moves ahead, as rather leveraging "experience" to know what's best next for a context, exactly like the "chess engine+HMM" did it in the past, but a lot less efficient.
I've had ChatGPT forget the current state of things with other stuff too. I asked it to do some web code, and it kept forgetting what state the files were in. I hear that some are better like Claude with access to a repo, but with ChatGPT even if you give it the current file as an attachment it often just ignores it and carries on blindly.
Yup, they currently have very limited context windows.
And it's also a case of "wrong tool for the wrong job". Keeping track of very large code bases is well within the range of much simpler software (e.g.: the thing that powers the "autosuggest" function of your IDE which is fed from a database of all functions/variables/etc. names of the entire database).
For code, you would need such an exhaustive tool to give the list of possible suggestion and then the language model to only predict which from the pre-filtered list, rather free-styling it.
For chess you would need to have a special "chess-mode" training that is trained to always dump the current content of the board and the list of most recent turns' moves in the scratchpad between each turn, so that the current state doesn't fall out of the context. Best would be to do it like people did with HMMs a long time ago: have a simple actual chess software keep track of the board and generate a list of all possible next legal moves, and use the Markov model to predict from that pre-filtered list(*).
(*): That could be doable currently with a piece of software that automatically generates a prompt "The current status of board is: {board description}. The last few moves where: {history}. Chose the best move from: {list of legal moves}".
I know it scanned and consumed like.. all of the great Chess games ever played. It can only predict the next word, or move.
...and this has been already demonstrated eons ago using hidden Markov models.
(can't manage to find the website with the exact example I had in mind, but it's by the same guy who had fun feeding both Alice in Wonderland and the bible into a Markov model and use it to predict/generate funny walls of text).
That seems like the nature of LLM's. If I ever can coax ChatGTP to play a whole chess game.. I will let you know the results.
The only limitation of both old models like HHM and the current chatbots is that they don't have a concept of the state of the chess board.
Back in that example, the dev used a simple chess software to keep track of the moves and the board and generate a list of possible next moves, then uses the HMM on that pre-filtered list to predict the next best.
Nowadays, you would need the chat both at least have a "chess mode" where it dumps the state of the board into its scratch pad, along a list of the most recent moves by each play, so that it always has the entire game in context.
Otherwise they should both do roughly the same thing (try to predict the next move out of a model that has been fed all the history of Chess games ever played), but with insane levels of added inefficiency in the case of the chatbot.
Tire particulate.
Check the
i.e.: a rich European country.
So a country with a not to shabby public transport network.
Thus compared to, say, the USA: it's already doing quite some efforts toward shifting traffic away from personal vehicles and toward much more efficient transportation systems that have a lot less problems per passenger than private vehicles.
Public transport is the best solution to reduce travelling-related pollution, but can't cover 100% of cases.
EV are a "good enough solution" at reducing problems caused by people who *MUST* and *CANNOT avoid* driving cars.
Can you tell me which thing is not like the others
The one which can play close to your entire Steam library of PC games, thanks to SteamDeck, SteamOS and Proton.
How many weeks are there in a light year?
