[Discord]

Code generation can be highly sensitive to prompting and initial conditions. Geniz explores an evolutionary approach to code generation, where multiple candidate solutions converge on a solution over multiple generations.

Given an input description of a coding task, the system generates, tests, and ranks a multitude of solutions to find the ones that perform the best.

Geniz includes an interactive codegen app that allows humans to select candidate solutions and evolve them until they converge on a correct solution. The human serves as a flexible fitness function in selecting applicable tests and candidates to survive to the next generation.

Geniz combines recent developed code-gen approaches like AgentCoder, Reflexion, and Self-Consistency. The approach particularly excels with smaller models. We foresee this approach as an economical option for achieving high-quality results with lower hardware requirements.

Benchmark Geniz without human invention on HumanEval dataset.

Note: * is our reproduction.

• Code solution generation: Geniz generates a variety of potential solutions in parallel.
• Test generation and execution: After generation, the system tests each candidate against all test cases and ranks them based on test evaluation and output consistency.
• Candidate evolution: New solutions are generated based on the previous generation of candidates.
• Interactive human feedback: Users can tweak solutions, modify tests, and select candidates to steer the evolution of better candidate solutions.

Geniz aims to change the way people tackle coding challenges from platforms like LeetCode, TopCoder, and USACO. Rather than spending hours coding and debugging, users can focus on defining the problem's requirements and let the LLMs handle the implementation details.

Geniz represents the first practical application of local/smaller LLMs, enabling them to solve complex coding problems effectively. By running the LLMs locally, Geniz ensures privacy and allows users to benefit from the latest advancements in language models without relying on external services.

To get started with Geniz, follow these steps:

• Clone the repository git clone https://github.com/sudocode-ai/geniz.git
• Install the dependencies: cd geniz && pip install -e .
• (Optional) Prepare keys.json (See keys.json.example), or type in the UI.
• Run the webapp: cd src/geniz/example && python webserver.py

We welcome contributions from the community! If you'd like to contribute to Geniz, please follow our contributing guidelines. Contact person: Ning Ren, Alex Ngai and Randy Song.

Geniz is released under the MIT License.