The post about AI

But lets clamp that down right away, this is more about LLMs, and even more specificically, code generation using LLMs.

GitHub CoPilot, Cursor, ChatGPT, you name it, it’s everywhere in developer land. I’ve been using Github CoPilot (in the browser for now) to have it do some work for a while now and it’s been hit-and-miss. It’ll do the boilerplate stuff well enough but at the same time it’s very easy get served something that makes no sense.

My current workflow is roughly like this:

• Be very specific and explicit in what you want done.
• Read through the generated thing at least once fully and make sure I understand what the code will do.
• Work on refinements/improvements (this is almost a given for every attempt) and changes for things I don’t need or are just plain wrong.
• Ask for verification by means of tests that go with the previously generated thing.

At this point I’m usually happy with whatever was created and will integrate the generated code by hand into the final product, validating once more that whatever I’m integrating is actually the thing I want. I’m not yet convinced that whatever was generated can right go into the product I’m working on.

For hobby stuff and exploratory coding for ideas I generally let the LLM go wild and use whatever it brings. Speed and discovery at the cost of validation!

There’s more to explore here though, but I am no authority to make any calls on where this LLM trend will go. It might still improve a lot, or maybe we’re stalling out… I have no idea, but here goes!

Software… reviewer?

So I’ve been seeing articles from folks letting the LLM do all the work of fixing the bugs and writing the features. It sounds like living on the edge to me, but if it works it works. With enough validation, why not.

But what about the developer prompting their way to a working product? Are they still a “software developer”? Or is that what a “prompt engineer” (LMAO) is? It sounds more like “software reviewer” when the AI is doing all the work and the human in the loop is just there as a safeguard, only reviewing the outputs. That does not sound like a fun job to me.

Vibe coding

I guess an extension of “the AI does all the work”, is just going with whatever is given by the LLM and trying to make it work.

As mentioned already, I do this a lot for exploration, experimentation and hobby stuff. It’s for me only and I’m the only user. Vibe coding my way into any substantial production grade product sounds irresponsible to me. Maybe when we see more improvments in the models that are used for code generation but I feel this is flying way to close to the sun.

Another aspect to this is that if you do find a bug or problem in the code and fix it, how do you prevent the LLM from making the same mistake again? If this was a human at work there would be the possibility for learning and experience but the LLM is static. Can you prompt the problem away for the next run? Update (or change) the model? Sounds a bit problematic.

The AI-native developer??

I’ve seen this one come by a few times as well recently and I really don’t know where to go with this. One explanation says that this type of role (?) is using AI throughout the whole development lifecycle, but then we’re still generating code, docs, a pipeline, that’s not much different from what most LLM use looks like I think.

Another take I’ve seen is that the prompts used to create the product are the source of truth and not the code. This sounds a lot like vibe coding on steroids. I would love to see this in practice for anything substantial.

The ladder

One concern I see is that while we can ramp up the productivity by generating more code, being able to review code will become more important, but the ability to review code requires experience. If there’s no path to gain that experience through actual hands-on coding (because the LLM is doing all that work), how is a junior going to pick up the skills and experience to judge the output of an LLM? Where do seniors come from when there is no good way to gain practical experience and skills as a junior?

Security

Running code you didn’t write and might not fully understand from a source that can confidently make mistakes and/or make stuff up… I guess you can see where this is going. Dependencies you didn’t vet that are suggested by that same source? Nah…

Anyway…

This whole LLM thing is another (pretty great) tool in the toolbox. I still use regular web search for stuff but when it’s something code related I generally go straight for the LLM and write out the question and ask for a sample to look at.

A web search will sometimes follow up a piece of generated code when there’s some halucinated non-existing crap in the sample.

I don’t expect the use of these tools to go away, in their current state they’re great for boilerplate stuff and cranking out an idea in a very short time. If LLMs have peaked or not, I don’t know but it’s here to stay. Getting familiar with them and how to best use them will only add to your available tools.

Just some thoughts and questions that came to me when I was off on vacation for two weeks. No real answers, more questions, thank you for reading 😄