Skip to main content
  1. Posts/
  2. AI/

Can AI work outside of software development?

·2 mins

I think we are still light years away from applying it to just any job, unless the result is automatically verifiable.

With software, it makes a mistake, corrects it, runs the tests, corrects it again, and eventually finishes.

You clear the context, ask the same model to analyse what it just did, and it finds problems. You send it back to implement, it changes the code, then sees it needs to change the test. Then you repeat, and it finds that the test change makes no sense.

After five or twenty-five rounds, it converges. There is a good chance the whole thing works.

Maybe it is 80% quality. In a lot of software, that is fine. I have built versions of this, with completely automatic rounds, and the final review becomes a small part of the process.

We might even be getting to the point where people skip it, because the enshittification of software is now normalised. Look at Bending Spoons: perhaps 80% quality is more than the market expects.

The models still cost money, of course. But that can still be less than paying a team of four developers.

Legal work gets a lot of attention because legal things are also more or less codified. But it is not like you can test whether the answer is good.

You need an expert to review it. That person still has to study the case, understand whether the answer is right for that case, verify the references because, well, go trust it, then find the problems and send it back.

Once you have manual loops where someone has to study the context every time, how much effort did you save? Twenty per cent?

And if you still need sophisticated models, with lots of compute, which means energy, they will never be that cheap. So there is no big saving.

That is why AI works well where it can test itself. Software is almost absurdly well suited to it: tests, compilers, type systems, linters, CI.

Most jobs do not have that equivalent yet.


comments powered by Disqus