FYI the video/image doesn't appear to work on github but I found the link in the raw file: https://youtu.be/UCo7YeTy-aE
The video cleared up a lot of questions for me and helped me to understand how powerful this could be. the debugger could be immediately useful even on existing codebases.
do you see this being useful as a development tool for existing codebases or mostly (smoller) newer products?
yea Akshat from Modal also called out the debugger as the actually useful thing
i originally thought of it as a scaffolding tool. instead of making and maintaining specific, rigid starters, like create-react-app, or create-supabase-app, this is basically create-anything-app.
its horribly slow right now but give LLMs 1-2years of improvement and i think something like this will be the way to start out any project - develop a prompt in tandem with the AI, specifying everything and anything you want in the app... and abandoning the tool and taking over the codebase once you think you've tapped out the scaffolding capability.
that said i do agree the debugger may have more utility for existing codebases... however the main hurdle here is that both the 32k context from gpt4 and 100k context from anthropic are only in private beta. so will have to wait to see it fully play out.
"create-anything-app" is a great name, gets the concept across
it would be neat for to existing devtools to extend/wrap this. for example, supabase could ask "what are you building, and what do you want to build it with?", and then it could scaffold everything on top supabase. Or svelte could do the same, and it would generate a backend that svelte could sit on top of.
yeah i havent tried but it could probably do both sides of a full stack app. the key thing to add is ability to install any dependencies discovered along the way as that dep list will grow with the growth in specs
Hi HN! made this this weekend to scratch an itch - felt like with long context now available with anthropic i wanted a universal page summarizer. but i hate the obscure dance of making chrome extensions. so of course instead of doing that i yakshaved myself into making a thing that makes chrome extensions... and every other possible smol app i might want in future.
this is the result of say ~1 straight day of experimenting and prompting, and learning to do whole program synthesis with LLMs unburdened by any knowledge of prior art or literature.
learnings:
- *Markdown is all you need* - Markdown is the perfect way to prompt for whole program synthesis because it is easy to mix english and code (whether `variable_names` or entire code fenced code samples)
- turns out you can specify prompts in code in prompts and gpt4 obeys that to the letter
- *Copy and paste programming*
- teaching the program to understand how to code around a new API (Anthropic's API is after GPT3's knowledge cutoff) by just pasting in the `curl` input and output
- pasting error messages into the prompt and vaguely telling the program how you'd like it handled. it kind of feels like "logbook driven programming".
- *Debugging by `cat`ing* the whole codebase with your error message and getting specific fix suggestions - particularly delightful!
- *Tricks for whole program coherence* - our chosen example usecase, Chrome extensions, have a lot of indirect dependencies across files. Any hallucination of cross dependencies causes the whole program to error.
- We solved this by adding an intermediate step asking GPT to think through `shared_dependencies.md`, and then insisting on using that in generating each file. This basically means GPT is able to talk to itself...
- ... but it's not perfect, yet. `shared_dependencies.md` is sometimes not comperehensive in understanding what are hard dependencies between files. So we just solved it by specifying a specific `name` in the prompt. felt dirty at first but it works, and really it's just clear unambiguous communication at the end of the day.
- see `prompt.md` for SOTA smol-dev prompting
- *Low activation energy for unfamiliar APIs*
- we have never really learned css animations, but now can just say we want a "juicy css animated red and white candy stripe loading indicator" and it does the thing.
- ditto for Chrome Extension Manifest v3 - the docs are an abject mess, but fortunately we don't have to read them now to just get a basic thing done
- the Anthropic docs were missing guidance on what return signature they have. so just curl it and dump it in the prompt lol.
- *Modal is all you need* - we chose Modal to solve 4 things:
- solve python dependency hell in dev and prod
- parallelizable code generation
- simple upgrade path from local dev to cloud hosted endpoints (in future)
- fault tolerant openai api calls with retries/backoff, and attached storage (for future use)
The video cleared up a lot of questions for me and helped me to understand how powerful this could be. the debugger could be immediately useful even on existing codebases.
do you see this being useful as a development tool for existing codebases or mostly (smoller) newer products?