Developing Software with Generative AI

While this book is about integrating generative AI particularly LLMs into software applications, in this chapter, we’ll talk about how we can leverage LLMs to help in software development. This is a big topic; software development has been highlighted in reports by several consultancies, such as KPMG and McKinsey, as one of the domains impacted most by generative AI.

We’ll first discuss how LLMs could help in coding tasks, and I’ll provide an overview to see how far we have come in automating software development. Then, we’ll play around with a few models, evaluating the generated code qualitatively. Next, we’ll implement a fully automated agent for software development tasks. We’ll go through the design choices and show some of the results that we got in an agent implementation of only a few lines of Python with LangChain. We’ll mention many possible extensions to this approach.

Throughout...

Software development and AI

The emergence of powerful AI systems like ChatGPT has sparked great interest in using AI as a tool to assist software developers. A June 2023 report by KPMG estimated that about 25% of software development tasks could be automated away. A McKinsey report from the same month highlighted software development as a function, where generative AI can have a significant impact in terms of cost reduction and efficiency gain.

The history of software development has been marked by efforts to increase abstraction from machine code to focus more on problem-solving. Early procedural languages like FORTRAN and COBOL in the 1950s enabled this by introducing control structures, variables, and other high-level constructs. As programs grew larger, structured programming concepts emerged to improve code organization through modularity, encapsulation, and stepwise refinement. Object-oriented languages like Simula and Smalltalk in the 1960s-70s introduced new paradigms...

Writing code with LLMs

Let’s start off by applying a model to write code for us. We can use one of the publicly available models for generating code. I’ve listed a few examples before, such as ChatGPT or Bard. From LangChain, we can call OpenAI’s LLMs, PaLM’s code-bison, or a variety of open-source models, for example, through Replicate, Hugging Face Hub, or – for local models – Llama.cpp, GPT4All, or Hugging Face pipeline integrations.

StarCoder

Let’s have a look at StarCoder, which is a small model for code generation and quite capable of doing that. It is available at Hugging Face Spaces at this URL: https://huggingface.co/spaces/bigcode/bigcode-playground

This screenshot shows the model in a playground on Hugging Face Spaces:

Figure 6.3: StarCoder Models Playground

We can choose between different models: StarCoder, StarCoderPlus, and StarCoderBase. Please note that, as the description says, the StarCoder...

Automating software development

In LangChain, we have several integrations for code execution, like LLMMathChain, which executes Python code to solve math questions, and BashChain, which executes Bash terminal commands, which can help with system administration tasks. However, while useful for problem-solving, these don’t address the larger software development process.

This approach of solving problems with code can, however, work quite well, as we’ll see here:

from langchain.llms.openai import OpenAI
from langchain.agents import load_tools, initialize_agent, AgentType
llm = OpenAI()
tools = load_tools(["python_repl"])
agent = initialize_agent(tools, llm, agent=AgentType.ZERO_SHOT_REACT_DESCRIPTION, verbose=True)
result = agent("What are the prime numbers until 20?")
print(result)

We can see how the prime number calculations get processed quite well under the hood between OpenAI’s LLM and the Python interpreter:

Entering new...

Summary

In this chapter, we’ve discussed LLMs for source code and how they can help in developing software. There are quite a few areas where LLMs can benefit software development, mostly as coding assistants. We’ve applied a few models for code generation using naïve approaches and we’ve evaluated them qualitatively. In programming, as we’ve seen, compiler errors and results of code execution can be used to provide feedback. Alternatively, we could have used human feedback or implemented tests.

We’ve seen how the suggested solutions seem superficially correct but don’t perform the task or are full of bugs. However, we can get a sense that – with the right architectural setup – LLMs could feasibly learn to automate coding pipelines. This could have significant implications regarding safety and reliability. As for now, human guidance on high-level design and rigorous review seem indispensable to prevent subtle errors...

Questions

Please look to see if you can produce the answers to these questions from memory. I’d recommend you go back to the corresponding sections of this chapter if you are unsure about any of them:

1. What can LLMs do to help in software development?
2. How do you measure a code LLM’s performance on coding tasks?
3. Which code LLMs are available, both open- and closed-source?
4. How does the Reflexion strategy work?
5. What options do we have available to establish a feedback loop for writing code?
6. What do you think is the impact of generative AI on software development?

Join our community on Discord

Join our community’s Discord space for discussions with the authors and other readers:

https://packt.link/lang