But again that's backwards. I let my AI coding assistant write tests exactly so that I have a reproducible, deterministic ground truth that I can inspect for logic errors. If you let the AI do the testing directly, it's a toss-up if today it feels like it or it decides that your desired result is "everything green" so it'll just give you that answer without actually testing anything.

LLMs obviously aren't human. But that's not the point. The point is that you don't need to know a programming language to program a computer.

For example: Especially in the early days of smartphones, there were mountains of small tools that did something that's essentially a simple script. Egg timers, remind-me-tomorrow apps, tons of stuff that's basically a wrapper around a web search, and so on. Almost all of those are today something ChatGPT can do, and are easy enough to describe.

I've actually outsourced the first iteration of that to AI. :-)

At the end of coding sessions, I generall ask it to spin up an adversial agent whose task it is to find flaws, inconsistencies, mistakes, bugs, etc.

I'd still strongly recommend to anyone shipping AI-generated code to have a thorough human code review, but for a first pass at surface level, the AI can check itself if it has some kind of separation between the generator and the reviewer.

Personally, after some experimenting, I've concluded that:

• AI is good at implementing known algorithms.
• AI is good in creating code that flows straightforward from a design document.
• AI is good at code reviews and finding bugs.
• AI (unlike me) doesn't mind writing a few hundred tests.
• But, it tends to over-engineering simple tasks to the point of "why does Hello World need enterprise-software levels of abstraction?"
• It struggles with code that doesn't have a known solution.
• It's difficult to get it into a different lane once it started going down the wrong way.

What I use it for these days is rapid prototyping. Using AI I can make a functioning prototype of an idea I have in a few days instead of a few weeks. I can quickly test out an idea for a feature or change in my existing codebase.

What I also have been using it for a while is as a code-reviewer and for someone like me who is mostly working solo, that's an insanely valuable thing. Not just the bugs themselves, but also the ability to discuss code with someone. Basically, AI is a rubber ducky that replies.

Mickeysoft lost the plot again.

The game-changing element of LLMs and agents is that they have a non-computer-language interface - you can talk to them in plain English (or Chinese or whatever). This makes complex computer interactions available to a much wider audience than any previous computing invention. Adapting humans to the AI is getting that backwards. It's a sign you don't understand the first thing about why AI is so widely successful.

I predict this will be an abject failure and a year from now will quietly be pushed to the very back of the bottom-most drawer, never to be spoken off again (though it is getting crowded in there for M$).

Meanwhile, in the real world, everyone I've talked to about these things already has a skills file to teach the AI about their personal coding style. This is what actual humans came up with, independently several times, and it works: Adapt the AI to you. Plus we've been writing coding style guides for humans for a long time now, we've got plenty of experience doing that at least somewhat right.

Are those conversations and results kept in an extending prompt? How far back can your bot typically recall? Does the company charge for extra prompt (history) size?

Grade homework on completion and be done with it. Then schedule in-class quizzes and exams, where students are expected to show their work as well as the results and do it hand-written. That makes it hard to access chatbots, and even if students do the drudgery of transcribing the chatbot's output will discourage them (if they were willing to do that much work, they wouldn't be trying to skip the work using a chatbot). They may skate by on the homework, but the exam scores will show whether they actually learned the material or not.

> Explain to the AI what it got wrong and have it fix it. You could fix it yourself but that will just confuse the AI.

Please elaborate. Confuse for what purpose or context?

If you need to cancel your model subscription for some reason, does all the bot's knowledge of your shop's practices and patterns go away?

If that's a problem, then in-house or fully open-source models may be safer even if not as good initially.

"Back in my day bots draw 8 fingers per hand, you spoiled whipper-snappers have it down to 6!"

Now MAGAs will have to troll the old-fashioned way: type Fox talking points into social media.

Are you talking as a language per programming productivity, or per security?

Don flips faster than his hair in the wind.

not very bright, but patient and determined, happily manipulating humans to get its way.