Where I found these “agents” to be the most useful is expanding on documentation (markdown files and such). Create a first draft and ask it to clean it up.
How would the thing know what you were thinking and intending, what were the requirements, constraints, pre- and postconditions? How do you work around the risk that you end up with something that looks like useful information - but isn’t ?
Because you created a first draft. Your first draft should include all that info. It isn’t writing the whole doc for you lol, just making minor edits to turn it from notes into prose.
Without that? No clue, good luck. They can usually read source files to put something together, but that’s unreliable.
I find getting the LLM to either generate or rephrase documentation gives a distinctly worse result than doing it myself. I was in a hurry to document a new API from scratch recently and thought I’d try Copilot, but the results were overly verbose and sometimes inaccurate so I ended up rewriting all of it.
The LLM is best for boilerplate code that is easily predictable and verifiable. Beyond that it’s sometimes good for initial suggestions if you don’t know where to start with a tool, after which you can go to the actual documentation. But you’ll need to do that, because half the time the suggestions use nonexistent APIs and methods.
I have always thought that writing code is the easy part of being a developer. The hard parts are the parts management doesn’t appreciate: clarifying requirements, architecting new systems, translating business goals into something codable, letting egotistical know they’re not making sense without offending them, designing effective testing processes, persuading management to prioritize reducing technical debt, and integrating and maintaining existing systems. Maintenance is a huge part of the job that no one gives you credit for. Oh, and if you ever touch the front end, CSS.
How would the thing know what you were thinking and intending, what were the requirements, constraints, pre- and postconditions? How do you work around the risk that you end up with something that looks like useful information - but isn’t ?
Because you created a first draft. Your first draft should include all that info. It isn’t writing the whole doc for you lol, just making minor edits to turn it from notes into prose.
Without that? No clue, good luck. They can usually read source files to put something together, but that’s unreliable.
I find getting the LLM to either generate or rephrase documentation gives a distinctly worse result than doing it myself. I was in a hurry to document a new API from scratch recently and thought I’d try Copilot, but the results were overly verbose and sometimes inaccurate so I ended up rewriting all of it.
The LLM is best for boilerplate code that is easily predictable and verifiable. Beyond that it’s sometimes good for initial suggestions if you don’t know where to start with a tool, after which you can go to the actual documentation. But you’ll need to do that, because half the time the suggestions use nonexistent APIs and methods.
I have always thought that writing code is the easy part of being a developer. The hard parts are the parts management doesn’t appreciate: clarifying requirements, architecting new systems, translating business goals into something codable, letting egotistical know they’re not making sense without offending them, designing effective testing processes, persuading management to prioritize reducing technical debt, and integrating and maintaining existing systems. Maintenance is a huge part of the job that no one gives you credit for. Oh, and if you ever touch the front end, CSS.