Software engineering has been one of the industries most actively adopting AI. There are few, if any, companies in this discipline that aren’t looking at large language models (LLMs) in some way, shape or form. For most, however, the technology is predominantly used in the lowest step of our maturity model, ie the AI assistant.
The AI assistant is supporting individual engineers in their tasks across a variety of domains and can significantly improve their productivity. So, although there’s a lot more room for exploiting AI in software development, this first step is far from trivial in its value and in terms of the benefits it provides.
Specifically, we can categorize the use of the AI assistant into five main areas: code-related support, knowledge management, workflow, communication and cognitive support. The main and most obvious area where LLMs are applied is in code-related tasks. For example, tools like Cursor are concerned with code completion. Although this may seem like something akin to autosuggestion in word editors, in practice the tools go significantly beyond that, including suggesting boilerplate code, generating repetitive structures and even providing entire functions with some well-known functionality. In addition, LLMs can help with refactoring and debugging.
The second main area is knowledge management. LLMs are especially helpful in cases where they can provide just-in-time documentation and present it in an understandable way. In that sense, they can become a coach or mentor who helps engineers acquire new knowledge in an easy and context-specific fashion. LLMs can also help in exploring design or code alternatives. First of all, engineers may not even be aware of the existence of alternative approaches to a specific problem. And even if they do, it may not be obvious what the pros and cons are of each of the alternatives, especially in the specific situation where the alternatives are evaluated.
The third area has to do with workflows and productivity in general. This can entail several aspects, but some typical cases include structuring a task, quality assurance, code reviews and managing context in different projects. For complex tasks, for example, the breakdown into smaller, manageable chunks isn’t necessarily trivial. Here, LLMs can help identify units of work that can be solved relatively independently. Once code is written and generated, we need to test it. For this, LLMs can help produce relevant test cases. Often, it’s best practice to use different LLMs for code generation and test case generation, but that doesn’t do away with their usefulness in this context.
Code reviews are both the bane of existence of most engineers and, at the same time, an effective tool for ensuring code quality and conformance to coding standards. The challenge is that getting submitted code back without it having been approved and with suggestions to improve it is both annoying for the engineer and inefficient for the organization. LLMs can help pre-review code, identify issues and fix them before submitting the code.
Finally, many engineers work on multiple projects that they need to switch between. It’s easy to forget some or most of the other contexts, so tools that help engineers reacquaint themselves with a project can be very helpful. This is where LLMs can also play a role.
Communication is the fourth area where LLMs can be of assistance. Virtually all software requires documentation, but few engineers enjoy writing it. Being language computers, LLMs are surprisingly good at generating software documentation and can provide significant benefits.
A second communication aspect is explaining technical decisions to people who don’t have a technical background. Many engineers struggle to put in plain English why a system is structured as it is. However, LLMs can be quite helpful as they tend to be able to find suitable parallels and metaphors to explain the pros and cons of specific technical choices.
Finally, the fifth area is concerned with reducing cognitive load. Again, one of the benefits of many LLMs for coding is that they work in-line. Hence, engineers don’t need to swap between different screens and applications. This simplifies the workflow and reduces task switching.
Another main aspect that doesn’t get enough attention is the media, in my view, is the use of LLMs as a sparring partner. It’s easy to explore ideas that initially might seem harebrained, but that during the discussion turn out to be quite interesting and relevant. As engineers have egos as well, this often wouldn’t have happened with a colleague – if nothing else, because the colleague is busy too and doesn’t have the bandwidth to explore things extensively.
The first step on the R&D maturity ladder is engineers using LLMs and AI agents for personal productivity improvement. These AI assistants can help in several areas. Here, we discussed the five main ones: code-related support, knowledge management, workflow, communication and cognitive support. Proper use of your assistant can lead to a significant productivity improvement, if used well, but your mileage may vary depending on your specific context. Although the idea of an AI assistant may not be real for everyone yet, to quote Dave Waters, in a few years, artificial intelligence virtual assistants will be as common as a smartphone.
Want to read more like this? Sign up for my newsletter at jan@janbosch.com or follow me on janbosch.com/blog, LinkedIn (linkedin.com/in/janbosch) or X (@JanBosch).
