At least here in Boston Verizon pays rent on the land occupied by their poles, so they are paying for the right of way. Them being in arrears for said rent famously caused a catfight with the city government that delayed the rollout of FIOS for years.
Shipping running code is obviously always a top priority. Non shipping is bad, not running is bad/worse.
This is the kinda thinking that crashed two space shuttles. There are worse things than not shipping, even if it means the project gets scrubbed. Sometimes safety or durability has a higher priority than shipping. An older less efficient system that works is more valuable than a new one that is unreliable. A new capability isn't worth it if it gets you killed.
and loose conventions No idea what that is supposed to mean.
What I mean is that the following is from agile's founding document:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
Try telling the FDA you value "Individuals and interactions" over following a defined CGMP or "working software over documentation" and see how far it gets you.
rigorous requirements requirements in agile projects underly the same rigor than non agile projects - otherwise they are not "requirements".
If your design and implementation is subject to a very heavy, long, and expensive V&V process, you can not "spiral" on them. You can't freeze a design if your requirements are subject to customer change at the end of every 2 week sprint.
And in 20 years when those of us who are 40-50 are retired or dead and there is no pipeline of fresh talent that has spent 20 years becoming experts to guide the AI, then I guess society just collapses. Good job everyone!
It works fine when you actually have a skilled team. It essentially codifies how we used to work in the late 90s/early 2000s, before the web was hot and we were inundated by kids trying to make money.
It really depends. If you are working on a system that can get someone killed or financially ruin a large institution (or sometimes both), the precepts of the agile manifesto kinda fall apart. You don't want to prioritize speed, shipping, running code, and loose conventions over rigorous requirements, thoughtful design, and risk management when you are writing a clinical, financial, or on-mission military software system.
Throw them in the Hague maybe? Just a thought.
But a 65" monitor (assuming you can find one and afford it) makes a lousy TV.
I was responding to a parent that said that you should just buy a monitor instead of a smart TV. I was pointing out that monitors and TV fill different use cases and have different pricing scales so they aren't necessarily drop in replacements.
A monitor the size of a modern 65 inch TV would be prohibitively expensive, if you could even find one. Even so, monitors have limited support for home theatre HDMI features like HDR, ARC, and CEC compared to TVs
I think it is hard to really air-gap these model test environments because the agent basically needs a network connection to talk to the model under test. The model is probably running on a room full of server racks that might not be practical to take offline.
It's a pretty big difference. The agent only gets the tools you give it. Also, when talking about commercial systems, the agent is more than just the model and harness. There's also an API layer that includes a system prompt the user has no control over that steers the model. There are also often watchdog processes (which may be agents in their own right) that are examining inputs and outputs for potentially dangerous or TOS-violating usage.
"...where the reward is for a correct answer..."
AI has no concept of reward, AI does not seek or desire and it cannot be incentivized.
"Reward" is just colloquial jargon for "reinforcing the model weights used to arrive at the correct answer". It's just a lot more convenient to say. We use that term because it is analogous to the how we think our own brain's reward system uses dopamine to reinforce neuron connections involved in certain behaviors. e.g. a toddler eats their vegetables, you praise them, brain releases dopamine, the neural links involved in that behavior get reinforced.
This is why systems like this require human oversight.
This is why systems like this should never be allowed to be built in the first place and should be destroyed wherever they are found.
Can you even trust locally deployed models? Maybe if they're completely isolated from the outside, but remember that OpenAI and Anthropic like to claim their models are SUPER HACKERS ABLE TO BREAK ANY SANDBOX now.
Models only accept input, convert it into tokens, perform some probabilistic match functions, and emit tokens. You need to hook them up to a piece of software that interprets those outputs as commands and feeds the results back to the model to give it any kind of agency. This is an agent harness. The permissions you give an agent harness are on you. There is no reason you have to hook your harness up to a full shell, especially if you are just trying to do some math research.
Also, those scenarios where the agents "broke out" were scenarios where OpenAI deliberately disabled safety controls and instructed the agents to do hacking attacks on target test systems. The tests got out of hand and escaped containment, but agents don't just "go rogue" for no reason: they were deliberately put into a "hacker" mode. There was a lack of adequate monitoring and sandboxing around the test environment. The test environment wasn't completely wiped between runs. Allowing artifacts to persist between runs meant that agents on subsequent were able to bootstrap towards escape earlier than previous generations. The fact that the test network had network access to the system where the API keys were stored for the monitoring system meant the rogue agents were disable the watchdog. That and the fact that the test network wasn't air-gapped were clear failures.
Yes, agentic harnesses can make destructive mistakes and you have to take care with tool permissions and sandboxing. But in the course of normal use, it's unlikely that a user running a local model risks accidentally hacking a random system. If you are doing something borderline like reverse engineering a remote proprietary REST API or something else clearly legal but not sanctioned, then yeah, be careful the agent doesn't decide that the easiest way to get the protocol is to break into the target. But that is on you to monitor what network calls are being made in that case. You need to put deterministic hard coded guards around what tools your harness executes. You are in absolute control of that. The local agent is just a dumb piece of plain software.
None of them did back in 2012 when Lyft and Uber were getting big, at least not where I was.
Part of the job of the police is to prevent crime. They don't have to wait for the angry ex-boyfriend stalker to hurt someone, they can have a conversation before that to prevent things from going too far.
It absolutely is not. Courts have ruled again and again that the police have no duty to protect. If there is a stalker ex-boyfriend, you get a restraining order. Then if they get close to you, there IS a violation of the law for police to act on. If you are not suspected of a criminal act, the police have no business detaining you, full stop.
"I say we take off; nuke the site from orbit. It's the only way to be sure." - Corporal Hicks, in "Aliens"
