We word-pattern-generator skeptics stay winning.
For what it’s worth, I don’t personally hold the view that AI tools turn everything into a triviality.
My overall experience so far is they are tools that are worth using rather than not. I think there is only a facade of intelligence, but it just so happens that the facade is very useful anyway.
They still have to be guided and used carefully, and like I’ve mentioned, we wouldn’t blindly accept anything that was AI generated. Even the very enthusiastic people like David would acknowledge that I think.
8 Likes
A rather bold statement. As someone who uses AI in various aspects of my professional and hobby development life, it is indeed a time saver if you know how to use it properly. It took me several months to become highly effective at using it, mostly learning the various models’ limitations and understanding when you don’t need to over explain things. Like any tool, it can be used improperly and become a burden. In the end it has made me a better and more effective programmer. I don’t need someone’s research to tell me, I’m using it to my advantage, daily. It can, quite literally, take weeks worth of manual effort and turn it into an afternoon.
Think about it this way. What are we trying to do when programming? We’re trying to take an abstract thought from our squishy organic brains and turn it into a logical model that can map to silicon. That’s it. Every software language is a portal where we as humans attempt to map our thoughts to logic. Using these LLM based agents is no different from going from assembly to C++, there just isn’t syntax to learn, which is a huge advantage. I no longer need to master 100 programming languages, I only need to understand concepts, have a logical train of thought, and be able to debug/understand the output.
So in my workflow, I first write a very detailed specification of exactly what I want to build, thinking of all the caveats, focus points, and identify potential problem areas. It is what I would do if I was programming by hand anyway. But instead of going and figuring out how to translate this functional requirements list into language XYZ, I feed it into an AI agent to do it for me. Then I go through line by line, understanding what it did, iterate, write tests to prove correctness, and go on to the next step. Instead of being a code monkey, I become a verification engineer.
If you don’t do the work in a careful thought out way, you’re just throwing dice. Garbage in, garbage out still applies, only now it is amplified. Used improperly, absolutely, I agree, AI is worse. To your point however, is AI the problem in these studies? Or are these programmers simply not adept in using them? It is a different skill set. Some of the most “experienced programmers” I’ve met over the years become very good at writing code, but terrible at clearly thinking about what they are doing at a high level. They just hack on it until they get what they want. These folks are not going to be successful using AI until they learn this new skillset.
8 Likes
I understand that developers, skilled developers believe that these tools help, and I watch the research that contradicts this. Which should I believe?
I’ve been seeing about one study each week recently showing that the claims for AI tools are overblown, that they frequently do not meet the claims or expectations, and that they have actual harmful effects on those using them.
I prefer research to anecdotal claims and will continue to monitor this.
I wouldn’t expect you to believe either, to be honest. I was in your camp just about a year ago, as was everyone I worked with. Finally I decided to try it myself. At the end of the day, I don’t give a $#!t about what some researcher or even my coworker says, its how it affects my ability to do the job at hand, and for me, it has been a very powerful tool that I can leverage. So from one developer to another, I suggest you actually try some of these tools. There are lots of free ones available. Given the proper constraints and stimulus, I think you would be surprised how far these things have come in just the last few months.
4 Likes
That last statement from David in this screenshot is exactly what I’ve experienced. He said it much more eloquently than I did 
5 Likes
I’d say generally that is true. The claims are overblown. However, that doesn’t mean the tools aren’t useful at all.
I’ve recently really enjoyed my experience with Claude Code. It’s a different workflow from, e.g., Cursor. It prompts for verification at every step it takes, giving you the opportunity to intervene and nudge it in the right direction if you need to. For me that is so much better than getting a big diff at the end.
Plus, with Claude Code I can go back to using Neovim rather vscode, which is so much better because that was what I was used to before. I love having the snappiness of Neovim back and all the other stuff I love about that.
5 Likes
I’d believe the skilled developers telling you they help/improve/speed up their work. Studies can be easily manipulated to get a desired result. More devs in here appear to use it than not. You remind me of my father when I started working with him and started introducing him to the new generation of power tools, inverter welders that weigh 19kg instead of 50kg, impact drivers rather than manual ratchets and screw drivers, he was adamant they were not an improvement and now has a van full of them. Maybe it’s not for you but that doesn’t mean it doesn’t work or cannot work.
1 Like
This is an irrational debate with fans of tools they ‘enjoy’ using ‘feeling’ that they make them more efficient, while research contradicts this time and again.
What David said is something I’ve been aware of and pointing out for a long time. The work of a developer is in understanding, analysing, specifing, coding and iterating. If you use a tool to do one part of this, you will have less understanding of the result, and less idea of whether it actually works. If you do the final stage yourself - for me a very enjoyable one - you have a greater understanding and will be more effective in future because your learning encompasses the whole process.
A compiler is also taking load off a programmer, but we know what a compiler does and can trust it to tell the machine exactly what we tell it to do. If you use an LLM, you cannot trust that, and if it doesn’t do what you tell it, you have problems that you don’t get doing the coding yourself, and you don’t learn as much. You reduce your understanding of the solution produced and your learning about different ways of doing this for future work.
These are not hard things to reason about and research has backed up my expectations about this. People can keep telling me that for them the research is wrong, and maybe so, but it isn’t a convincing argument to me.
There are a lot more problems with LLMs for code than this but I’m not going to keep bashing this, I will stick to pointing out what the research says.
1 Like
This I agree with as it is what I found even with my small amount of experience using AI. Note it says “familiar to them”
BUT what I referred to elsewhere was one aspect being helpful amongst the less helpful was when working in new/unfamiliar codebases and ability to introduce a person to new libraries instead of spending a lot of time searching them out.
Once that is known the rest is like tutoring a 1st year student in simple coding practise. IE a lot of time to tease out good code, if at all.
To me this indicates using LLMs can be helpful if used in the correct way.
As to calling them AI, well if I go back to my AI computer science classes of the late 70’s (or was it very early 80’s) then the LLMs are considered in the category of AI sciences. Even have a text book back from the late 70’s which actually describes the (early algos) that the LLMs use.
But as to referring to them as the movies style of AI then they are nothing like that, they do not have intelligence or real reasoning. Their reasoning is really just a different algo applied to essentially decision tree processing done by new methods like neural style processing.
1 Like
Yes, I read that, and it doesn’t really change much because using an LLM on code you are not familiar with makes some things worse. Anyway, I’ve made my point.
language transformer models with attention head? oO …
aaaah - early algos … you mean next token prediction as a concept?
I think it’s not black and white …
turning my code from a Jupyter notebook (an environment where I can just fire async await statements) into a runnable python script with event loop creation stuff that can run my code error free over there:
literally took me 10 seconds and saved me some minutes of lifetime … not much … but still nice
2 Likes
Language is really only a small part of the AI sciences. But it is its own specialised area and decades to get to a point where they had a “breakthrough” and it could actually get a high success rate in “understanding” language.
But we had language models all through this century and some even deployed in these phone response systems and everyone hung up on them because they were so bad. Language has to reach a certain point before it is considered useful as a tool and once that happened the industry went to commercialising it. OpenAI was one of those.
LLMs can be a tool, like a hammer, you don’t use it as a fine screwdriver on your watch, or screwing in that circuit board. Its more of a search engine that can understand your language. But for actual coding, its really only good for giving examples that you would normally search for elsewhere taking a lot longer than the llm does. But for good coding it is really hit and miss and most times its a miss. Its a case of “use correctly it can help save time when searching for those illusive libraries and the interfaces for them” and provide the “code examples” that one would normally spend hours looking for. Of course you still evaluate them whether LLM provided it or your searching found them.
As you say its not black/white and the study parameters can really skew the results. Was it seasoned developers knowing its limitation finding it 20% time waster or was it developers who think they are seasoned relying heavily on llms. Or a mixture? Will make a big difference in the results. If you want it to do your coding and you fix that, then its a time waster.
3 Likes
and I don’t understand why we’re investing time in this discussion … it’s pretty simple - either it helps or it doesn’t … and everyone can just decide for himself if he wants to use it or not …
5 Likes
To a certain extent.
If you want to ‘manually’ code up your own personal projects at least partly because you enjoy the act of hacking out code yourself–go for it. However, if you are trying to make a living as a developer and you aren’t using these tools, you’re likely going to be committing career suicide. Maybe in 5 years time we’ll be proven wrong about that and we’ll discover LLMs have wreaked so much havoc that we came to the conclusion they were not worth using, but personally I don’t see that happening.
8 Likes
Just like how the llms have entered in the last few years, the coding side of “AI” will improve over time as people use it and feedback comes in.
To think that coding “AI” will stagnate at the level it is today is backward thinking. It will improve over time and as long as we continue to see it as a tool, maybe a swiss army knife in a few years, and not think of it as intelligent (like a human) then we should be able to benefit from it. Remember to a hammer everything looks like a nail.
4 Likes
if that’s the case this would be a decision too - right?
I wouldn’t go that far with a prediction … but code editing with LLM-supported IDEs is indeed significantly more than just next token prediction … and using ChatGPT now is already totally different to using it half a year ago …
…llms are trained to use tools (they don’t try to calculate themselves anymore … they can fire web searches, process results with other tools, …) and those tool calls then are chained together to get agent flows …
…when I started out testing ChatGPT it was a total waste of time … the output looked nice but was rubbish … then it was pretty cool in generating simple docker-compose files for starting some apps as stacks … now it’s pretty mature and doesn’t necessarily write the best code … but it writes code that is mostly error free (formally correct) …
1 Like
If you don’t have a mortgage to pay? 
5 Likes