CopilotChat - AI Code Generation with Test-Driven Development | Product Hunt
Try it Now

Copilot Chat: Code Generation with Test-Driven Development

A test-driven, multi-stage, iterative flow that enhances the code quality of LLMs.

Introduction

AI-driven code generation tools like GitHub's Copilot have revolutionized the way developers write code. However, ensuring the generated code consistently meets specific requirements and passes all test cases can be a time-consuming process.

CopilotChat is designed to simplify code generation with the Test-driven development (TDD) process. Developers can easily specify their test cases and let CopilotChat generate code that not only fulfills their needs but also passes through validation.

How it works

Developer1. Define Test Cases
  • Define inputs and expected outputs.
  • Provide an optional requirement description.
LLM2. Code Generation
  • LLM generates code based on the test cases and requirement description.
Copilot Chat3. Validation
  • Copilot Chat validates the generated code.
  • If a test case fails, Copilot Chat iteratively interacts with LLM to refine the code until all test cases pass.
  1. Define Test Cases

    Developers begin by defining their test cases, providing inputs and expected outputs. For instance, when parsing a GitHub URL into a structured object, developers would specify test cases like:

    input: "git+https://github.com/group1/name1.git"    expectedOutput: {groupName: "group1", projectName: "name1" }
input: "git+ssh:https://[email protected]/group1/name1.git"  expectedOutput: { groupName: "group1", projectName: "name1" }
  2. Code Generation

    CopilotChat then takes charge by interacting with LLM to generate the JavaScript code.

  3. Validation

    The generated code is automatically validated against the provided test cases. If any test case fails, Copilot Chat iteratively keeps working with LLM to refine the code until all tests pass.

Example Use Cases

  1. Parsing Git URLs

    Developers can easily generate code to parse Git URLs and extract relevant information like the group and project name.

  2. String Manipulation

    From parsing css color string to capitalizing every word in a string, CopilotChat excels at generating code for a variety of string manipulation tasks.

Revolution of LLMs Code Generation

Inspiration from TypeChat

Similar to TypeChat, CopilotChat revolutionizes the way developers interact with code generation of LLMs. The tool allows developers to define test cases that serve as a representation of their coding intents. Much like TypeChat's approach to defining types, CopilotChat takes these test cases as a guide and handles the intricate process of generating code that aligns precisely with the specified requirements.

From Prompt Engineering to Flow Engineering

CopilotChat also serves as a simplified JavaScript implementation of the concepts presented in the paper "Code Generation with AlphaCodium: From Prompt Engineering to Flow Engineering." It's a testing-oriented flow centered on an iterative flow, which repeatedly runs and refines the generated code against input-output tests.