Nice job on this, it’s a really interesting approach. I’ve been developing an open-source coding agent over the past year, and RAG just wasn’t working at all. I switched to a repo map approach (which sounds similar to what aider is doing) and that helped a bit but still wasn’t great.
However, a few weeks ago I built an agent that takes in a new GitHub issue and is given a variety of tools to do research on the background information to complete the issue. The tools include internet searches or clarifying questions to ask the person who wrote the ticket. But the most useful tool is the ability to look at the codebase and create a detailed markdown file of various files, explanations of what each file does, relevant code samples or snippets from the files, etc..
It’s still early, but anecdotally I’ve seen a huge increase in the quality of the code that uses this research as part of the context (along with the repo map and other details). It’s also able to tackle much more complex issues than it could before.
I definitely think you’re on to something here with this wiki approach. I’ll be curious to dig in and see the details of how you are creating these. Here is my research code if you’re interested: https://github.com/jacob-ai-bot/jacob/blob/feature/agent/src...
We architected JACoB to make it relatively easy to add new frameworks. There are templates that dynamically build prompts based on the languages and tools defined in a jacob.config file that lives in your repo. Then you can just add new files with custom instructions for the new language. Here's the prompts folder with examples for the technologies we support today: https://github.com/jacob-ai-bot/jacob/tree/main/src/server/p...
I'm personally not a Rails dev so I haven't tried it, but my intuition is that Rails would actually be a good candidate to work well with JACoB due to the relatively standard conventions that most projects use.
Awesome work! I had seen this a few years ago, then I was looking for your site earlier this week but had forgotten the name. So great to see that you’re still continuing to post new designs.
I've been experimenting with using LLMs to map Figma designs directly into a working production-level codebase. There's quite a bit of compression you need to do in order to convert the raw Figma JSON into a format that an LLM can understand, but overall it actually performs quite well. Here's a quick demo: https://www.youtube.com/watch?v=s9JRBw7kR9g
Yes, this is a very similar technique that I have been using and it works great. One suggestion of something that worked well for me was to use safeParse instead of parse. Then if it doesn’t pass validation, you can retry by passing in the JSON object and the validation error messages. You could also use tricks like starting with a smaller model, then try larger models if you hit a validation failure. Not a great approach for real-time chat but very useful for when you need high-quality results.
A few weeks ago, I saw Alex Kolchinski's post about using the Shortcuts app to talk to GPT-3. I was intrigued, so I started hacking around with a few different scenarios and quickly found myself using them on a daily basis. Shortcuts are pretty useful for automating tasks, and the direct integration with the OS makes it easy to copy/paste and run keyboard shortcuts in any app or website.
I shared a few of my Shortcuts with our team, and others started remixing them and creating their own to share with us. We had so much fun making and using them that we decided to create PromptPlays, a website to share them with the public.
PromptPlays makes it easy to share and remix Shortcuts. You can copy/paste and run keyboard shortcuts in any app or website, and the marketplace allows you to browse and remix Shortcuts for your own purposes. All of the Shortcuts are available for free (BYO key).
We hope that PromptPlays will become a hub for people to share their ideas and create useful automations. Would love to hear your feedback on ways to improve or other ideas for ways this could be used.
I read everything on your website, but I still lack a fundamental vision of what this can do and how to use it. I need to see a few worked practical examples. Like many of us in tech, I suspect that you are so close to your product that you lack the more distant perspective necessary to devise meaningful examples. Professional help devising and writing these examples is probably necessary. I imagine the payback for this would be huge. Good luck with your project.
Great feedback, thank you. Here’s a video we put together to explain it in more detail. I think it’s a good idea to put something like this in the site: https://m.youtube.com/watch?v=YE62bfSq97M
Acquired is definitely one of the better tech podcasts out there. Been listening since season one, lots of good lessons learned but also super entertaining and fun.
However, a few weeks ago I built an agent that takes in a new GitHub issue and is given a variety of tools to do research on the background information to complete the issue. The tools include internet searches or clarifying questions to ask the person who wrote the ticket. But the most useful tool is the ability to look at the codebase and create a detailed markdown file of various files, explanations of what each file does, relevant code samples or snippets from the files, etc..
It’s still early, but anecdotally I’ve seen a huge increase in the quality of the code that uses this research as part of the context (along with the repo map and other details). It’s also able to tackle much more complex issues than it could before.
I definitely think you’re on to something here with this wiki approach. I’ll be curious to dig in and see the details of how you are creating these. Here is my research code if you’re interested: https://github.com/jacob-ai-bot/jacob/blob/feature/agent/src...
And here’s an example of the research output (everything past the exit criteria section): https://github.com/kleneway/jacob/issues/62