> In the past year, agent harnesses crossed the “can it be done” rubicon.
Brother, I'm still in "Can you get it right?"-mode. What am I doing wrong? (Rhetorical, but advice welcomed).
MattGaiser•Aug 16, 2026
What is “it” specifically and what languages are you using?
al_borland•Aug 16, 2026
I’ve found some success is small projects, with limited scope, in a greenfield.
I’m terrified to attempt agentic anything in the repo my job actually cares about. I triggered it once by accident, when the agent was first rolled out and enabled by default… it broke everything. Now I just use ask mode, and even that is wrong half the time, and once it goes wrong it just keeps getting worse.
I saw a post from Dave Plumber who vibe coded up a new cross platform task manager. He said his spec document for the AI was 107 pages long. So maybe what I’m doing wrong is not giving the AI a literal novel of spec.
applfanboysbgon•Aug 16, 2026
> He said his spec document for the AI was 107 pages long.
This sounds like programming but with extra steps that make it take longer with less reliability.
0x696C6961•Aug 16, 2026
Ikr, at that point the code itself is a better way of encoding the information.
simonw•Aug 16, 2026
Tell it to use red/green TDD and start things off with an already configured test suite, maybe with a single test that asserts 1+1==2.
Make sure it know how to run the tests before it starts writing any additional code.
Then set it a clear goal.
slopinthebag•Aug 16, 2026
Basically all the examples of LLM's building impressive things have been because they have human written tests to base the implementation on. If you have an LLM write the tests the results are far less impressive or valuable.
bharatsuthar•Aug 16, 2026
Yes and LLMs are known to cheat on tests written by them.
slopinthebag•Aug 16, 2026
It's not always cheating either. They aren't intelligent, so they don't actually understand the purpose of the tests or can build them to define the actual semantics of the problem space. It's literally just next-token prediction based on the codebase and prompt. Cheating implies that they have agency, and ironically agents don't.
mw888•Aug 16, 2026
You're appealing to ambiguity. All you've said is you have failed—how is anyone supposed to know what went wrong?
jaggederest•Aug 16, 2026
I'd be happy to screenshare with you if you like, we can work on something trivial or open source. Half an hour should be more than enough to see whether you're doing anything obviously self-sabotaging.
dosisking•Aug 16, 2026
There are two 'camps' with respect to AI.
One camp already knows that Neural Nets don't work and are a dead end.
The other camp hasn't yet figured out that Neural Nets don't work, but are convinced that they do (or eventually will), because they think everything always improves over time in a linear fashion.
mortalapeman•Aug 16, 2026
With generated code, the directory structure, interface design and general state management is usually a haphazard mess. Even with the best frontier models. But what really gets me is the model often tries to make assumptions for me that I didn't specify in the prompt. Subtle things like which error states are "oh shit we need to bail" vs "this isn't a deal breaker." Sometimes it will ask, but more often than not it will just make a decision and it's often the wrong one. If I don't have a fully kitted out test suit and a good type checker to verify the final product against, the the whole looping thing is just useless to me and I'm back to reviewing every line of code it puts out and having to draw on my years of architecture experience to make sure we don't build a giant pile of trash.
Gigachad•Aug 16, 2026
Because they are designed to be used by managers who don't know how to answer these questions and don't want to be asked them. Just have the magic answers box pick something.
slopinthebag•Aug 16, 2026
They're RLHF'ed to an inch of their lives to be able to one-shot complete tasks, since requiring human input defeats the purpose of being able to replace the labor force.
But once the insanity ends LLMs will be packaged as tools for developers to use to boost their productivity, and we'll consider them as we do IDE's and debuggers and stuff. But we have to get through this hype cycle first.
siva7•Aug 16, 2026
wake up, slopinthebag. wake up..
bluegatty•Aug 16, 2026
The generated code is fine at the functional level, the directory structure is usually the standard pattern for the given type of project.
The error types and codes, it will produce to spec.
If you type 'make me that thingy' - yes, it's probably not going to do what you want, but if you give it spec and guidance, it usually will.
The 'interface design' ... not very good though.
yoz-y•Aug 16, 2026
It absolutely hates code that would crash or error in any circumstance. So it adds a ton of dubious fallbacks.
aryehof•Aug 16, 2026
> But what really gets me is the model often tries to make assumptions for me that I didn't specify in the prompt.
This is a problem with your instructions, your specification. An LLM isn't a mind reader. It will attempt to succeed regardless of missing requirements and ambiguity.
Alien1Being•Aug 16, 2026
Surely this is a solvable problem.
If the average, mediocre software developer can address the issue of directory structure, interface design, general state management, edge cases and subtle assumptions it should be possible to train AI systems to address these issues .
Software development is not some mystical magical activity.
I remember people making similar arguments about autonomous driving...
chrisjj•Aug 16, 2026
> more often than not it will just make a decision and it's often the wrong one.
Let's not forget these chatbots rely on a random number generator to pick output options.
theteapot•Aug 16, 2026
> It helps to know that LLMs don’t “reason”. They predict ..
Semantics. Prediction is the training objective. The ability to reason can be, and very arguably is, an emergent property of that.
jayd16•Aug 16, 2026
Even if that was true, you'd have to still prove it has emerged.
krackers•Aug 16, 2026
What would be your test to determine that?
slopinthebag•Aug 16, 2026
Why would "reasoning" be an emergent property of prediction?
hsn915•Aug 16, 2026
How do you predict without reasoning?
slopinthebag•Aug 16, 2026
Where is the reasoning in linear regression?
danielbln•Aug 16, 2026
Where is the reasoning in synaptic transmission?
necovek•Aug 16, 2026
There isn't, which is exactly the point: we do not yet understand the fundamentals behind reasoning.
anon48293•Aug 16, 2026
Don’t we? We can build something which has all the output associated with reasoning.
I’d say we’ve figured out the fundamentals behind reasoning.
chrisjj•Aug 16, 2026
[delayed]
chrisjj•Aug 16, 2026
[delayed]
mw888•Aug 16, 2026
Predict multiple outcomes, induct across them, refine.
js8•Aug 16, 2026
There's a lot of reasoning in the training data.
hbcdbff•Aug 16, 2026
Why wouldn’t it?
complex_pi•Aug 16, 2026
Maybe it looks like reasoning, and maybe that's enough for some.
TheWrongGuy•Aug 16, 2026
By that standard, human brains don't either. Our externalizations of concepts like language or symbolic structure allow us to do so. In the parlance of our times, we built our own reasoning harnesses because our intuition lead us to do so.
bluegatty•Aug 16, 2026
"They’re foundationally incapable of always and consistently preventing prompt injection attacks. “Alignment work”, safety harnesses, and sandboxes all help to add barriers against the worst, but there are fundamental gap" ...
They seem to be very good at a lot of rudimentary best practices, more so than humans, but more accurately - if you run and audit pass with specific instructions ... they're very good at that.
I mean - it's what they're the best at which is applying 'fuzzy heuristics' in a mechanical way. If can describe issues concisely, the patterns, the styles, the rules then LLMs can very mechanistically and methodologically grind through them.
I don't even see how this is controversial - without getting into 'what their reasoning means' - we can all agree that their synthetic reasoning is pretty good at narrow scales, and they've been 'trained by compilers' and are extremely good at spotting common patterns.
If you back that up with a lot of tokens ... they excel.
Designing architecture, that's difficult, but hammering away at all the 'known-knows across a system' especially to identify things ... they're pretty good at that.
dmitrijbelikov•Aug 16, 2026
LLM is the new Excel
user43928•Aug 16, 2026
The article says what many here like to hear, but in my opinion the core arguments are false.
> Making software debuggable, maintainable, layered, and composable – that’s still quite a trick
Not really. I have been working on a mobile app for months, and I stopped even glancing at the code about two months ago.
150k LOC, around half of that in tests, and the AI still has no problem maintaining the code on my behalf.
Debuggable? It can add extensive instrumentation in seconds.
None of this requires expertise, prompting, or mention of TDD. It's the default.
Frankly I do not believe the author tried developing a large codebase fully agentic and without reviewing the code. I believe many here look at the code produced, deem it substandard, and go hands on.
> They’re foundationally incapable of always and consistently preventing prompt injection attacks
From Anthropic's article about the Auto mode:
> We commissioned an evaluation from a third party, Trajectory Labs, who tested different models within the latest publicly available versions of Claude Code and Codex as of July 17th 2026.1 They tested 72 indirect prompt injection scenarios held out from Anthropic
> In this evaluation, none of the 720 attack attempts succeeded against Claude Fable 5, Opus 5, or Sonnet 5 running auto mode. On the other hand, 5.83% of the attacks succeeded against GPT-5.6 Sol running Codex's Auto-review mode. Notably, this is greater than the 0.09% average attack success rate against our latest models running in bypassPermissions mode without additional safeguards. The tests showed a 19.03% attack success rate against GPT-5.6 Sol when running in Full Access mode
I'm sure someone is going to reply with how they do not trust Antrophic's research, but lacking other data, prompt injection appears to be largely solved already.
hbcdbff•Aug 16, 2026
How do you expect us to take your views on LLM code quality and durability seriously when a) you don’t even look at the code and b) you’ve only been doing this for two months?
user43928•Aug 16, 2026
I've been working on the app for four months, and I am clearly not talking about code quality.
I am talking about product quality and maintainability. Both are more than adequate.
I know this because I have worked on it for an estimated 300 hours. Has the author practiced a similar approach for even a week? I doubt it.
Krei-se•Aug 16, 2026
I work on my project for 2 years now and using an LLM always came back to bite me. Learning how something works is needed, slow and painful - but pain is gain.
If this works for you - awesome. Until it doesn't.
As always there is 0 code or link. All talk.
shakna•Aug 16, 2026
Prompt injection. Solved.
But accidentally breaking systems is not an issue either, obviously. Even though the system prompt asks for safety rails, and other prompts wouldn't accidentally violate that.
Alignment of the latest models is questionable, yes. That's a different topic.
For this particular gym incident, supposedly Opus 4.6 was used in OpenClaw, predating the current safety guardrails of Fable and co.
Alien1Being•Aug 16, 2026
AI generated code is like IKEA furniture.
IKEA furniture embodies many elements of good cabinet making but skips many nonessential elements. And does this more consistently than cabinet makers who can be bored, incompetent, depressed, burnt out, resentful, tired, having a bad day.
In the future AI code inevitably will embody most good software engineering practices. And will do this more consistently than software engineers who can be bored, incompetent, depressed, burnt out, resentful, tired, having a bad day.
Just look at the messages on HN or around you at your colleagues to see how mediocre the average software engineer is..
Today's IKEA is good enough for most people.
Tomorrow's AI coding will be good enough for most corporations.
Good enough to vastly reduce the need for fine craftsmen and women / software engineers.
Good enough to deskill those who call themselves cabinet makers / senior software engineers. These days the cabinet makers I personally know just do contract kitchens for project builders.
But IKEA is and AI will be, bad enough that at the high end with special requirements / taste / money / an inflated sense of self worth, some furniture makers still exist and thrive.
Perhaps 1% percent of current software engineers of today will be needed in the future when AI code inevitably has the ability to follow good software engineering practice.......
And as usual it will mainly be the mediocrities that remain ( so there is hope for you too ), with occasional islands of excellence.
hugodan•Aug 16, 2026
...ah the classic middle manager analogy of code is X, where X is nothing like code at all but is being used to drive a point that is just standing on poor grounds.
keep it, we have been through "is like building a house", "like following a recipe", "like a living organism", "like $SOMETHING_WITH_COMPONENTS", etc...
we can handle your IKEA furniture, thanks you for your contribution
amelius•Aug 16, 2026
Having not been formally verified, almost all software today feels cheap. Maybe an AI can change that at some point.
ChrisGreenHeur•Aug 16, 2026
Yay let’s lock ourselves into the formal verification toolsets, so that we can never use new language features again.
Almondsetat•Aug 16, 2026
It seems people have forgotten that engineering is actually a thing, and there are many aspects of SWE that can be dealt with like a real engineer. As an example, if you have a producer-consumer system, you can model it using a queue, and you can use queue theory to calculate what it would take to achieve certain guarantees. Hard, cold, calculations, like a civil engineer evaluating a structure. I chose to be a SWE, in the real sense of the word, because I believe this kind of approach and expertise in missing from many companies and software projects.
amelius•Aug 16, 2026
But do the same SWE fundamentals apply if the one doing the programming is many times smarter than us?
9 Comments
Brother, I'm still in "Can you get it right?"-mode. What am I doing wrong? (Rhetorical, but advice welcomed).
I’m terrified to attempt agentic anything in the repo my job actually cares about. I triggered it once by accident, when the agent was first rolled out and enabled by default… it broke everything. Now I just use ask mode, and even that is wrong half the time, and once it goes wrong it just keeps getting worse.
I saw a post from Dave Plumber who vibe coded up a new cross platform task manager. He said his spec document for the AI was 107 pages long. So maybe what I’m doing wrong is not giving the AI a literal novel of spec.
This sounds like programming but with extra steps that make it take longer with less reliability.
Make sure it know how to run the tests before it starts writing any additional code.
Then set it a clear goal.
One camp already knows that Neural Nets don't work and are a dead end.
The other camp hasn't yet figured out that Neural Nets don't work, but are convinced that they do (or eventually will), because they think everything always improves over time in a linear fashion.
But once the insanity ends LLMs will be packaged as tools for developers to use to boost their productivity, and we'll consider them as we do IDE's and debuggers and stuff. But we have to get through this hype cycle first.
The error types and codes, it will produce to spec.
If you type 'make me that thingy' - yes, it's probably not going to do what you want, but if you give it spec and guidance, it usually will.
The 'interface design' ... not very good though.
This is a problem with your instructions, your specification. An LLM isn't a mind reader. It will attempt to succeed regardless of missing requirements and ambiguity.
If the average, mediocre software developer can address the issue of directory structure, interface design, general state management, edge cases and subtle assumptions it should be possible to train AI systems to address these issues .
Software development is not some mystical magical activity.
I remember people making similar arguments about autonomous driving...
Let's not forget these chatbots rely on a random number generator to pick output options.
Semantics. Prediction is the training objective. The ability to reason can be, and very arguably is, an emergent property of that.
I’d say we’ve figured out the fundamentals behind reasoning.
They seem to be very good at a lot of rudimentary best practices, more so than humans, but more accurately - if you run and audit pass with specific instructions ... they're very good at that.
I mean - it's what they're the best at which is applying 'fuzzy heuristics' in a mechanical way. If can describe issues concisely, the patterns, the styles, the rules then LLMs can very mechanistically and methodologically grind through them.
I don't even see how this is controversial - without getting into 'what their reasoning means' - we can all agree that their synthetic reasoning is pretty good at narrow scales, and they've been 'trained by compilers' and are extremely good at spotting common patterns.
If you back that up with a lot of tokens ... they excel.
Designing architecture, that's difficult, but hammering away at all the 'known-knows across a system' especially to identify things ... they're pretty good at that.
> Making software debuggable, maintainable, layered, and composable – that’s still quite a trick
Not really. I have been working on a mobile app for months, and I stopped even glancing at the code about two months ago.
150k LOC, around half of that in tests, and the AI still has no problem maintaining the code on my behalf.
Debuggable? It can add extensive instrumentation in seconds.
None of this requires expertise, prompting, or mention of TDD. It's the default.
Frankly I do not believe the author tried developing a large codebase fully agentic and without reviewing the code. I believe many here look at the code produced, deem it substandard, and go hands on.
> They’re foundationally incapable of always and consistently preventing prompt injection attacks
From Anthropic's article about the Auto mode:
> We commissioned an evaluation from a third party, Trajectory Labs, who tested different models within the latest publicly available versions of Claude Code and Codex as of July 17th 2026.1 They tested 72 indirect prompt injection scenarios held out from Anthropic
> In this evaluation, none of the 720 attack attempts succeeded against Claude Fable 5, Opus 5, or Sonnet 5 running auto mode. On the other hand, 5.83% of the attacks succeeded against GPT-5.6 Sol running Codex's Auto-review mode. Notably, this is greater than the 0.09% average attack success rate against our latest models running in bypassPermissions mode without additional safeguards. The tests showed a 19.03% attack success rate against GPT-5.6 Sol when running in Full Access mode
I'm sure someone is going to reply with how they do not trust Antrophic's research, but lacking other data, prompt injection appears to be largely solved already.
I am talking about product quality and maintainability. Both are more than adequate.
I know this because I have worked on it for an estimated 300 hours. Has the author practiced a similar approach for even a week? I doubt it.
If this works for you - awesome. Until it doesn't.
As always there is 0 code or link. All talk.
But accidentally breaking systems is not an issue either, obviously. Even though the system prompt asks for safety rails, and other prompts wouldn't accidentally violate that.
https://www.abc.net.au/news/2026-08-10/ai-assistant-hacks-gy...
For this particular gym incident, supposedly Opus 4.6 was used in OpenClaw, predating the current safety guardrails of Fable and co.
IKEA furniture embodies many elements of good cabinet making but skips many nonessential elements. And does this more consistently than cabinet makers who can be bored, incompetent, depressed, burnt out, resentful, tired, having a bad day.
In the future AI code inevitably will embody most good software engineering practices. And will do this more consistently than software engineers who can be bored, incompetent, depressed, burnt out, resentful, tired, having a bad day.
Just look at the messages on HN or around you at your colleagues to see how mediocre the average software engineer is..
Today's IKEA is good enough for most people.
Tomorrow's AI coding will be good enough for most corporations.
Good enough to vastly reduce the need for fine craftsmen and women / software engineers.
Good enough to deskill those who call themselves cabinet makers / senior software engineers. These days the cabinet makers I personally know just do contract kitchens for project builders.
But IKEA is and AI will be, bad enough that at the high end with special requirements / taste / money / an inflated sense of self worth, some furniture makers still exist and thrive.
Perhaps 1% percent of current software engineers of today will be needed in the future when AI code inevitably has the ability to follow good software engineering practice.......
And as usual it will mainly be the mediocrities that remain ( so there is hope for you too ), with occasional islands of excellence.
keep it, we have been through "is like building a house", "like following a recipe", "like a living organism", "like $SOMETHING_WITH_COMPONENTS", etc...
we can handle your IKEA furniture, thanks you for your contribution