I have a small fish script I use to prompt a model to generate three commit messages based off of my current git diff. I'm still playing around with which model comes up with the best messages, but usually I only use it to give me some ideas when my brain isn't working. All the models accomplish that task pretty well.
Interesting idea. But those say what’s in the commit. The commit diff already tells you that. The best commit messages IMO tell you why you did it and what value was delivered. I think it’s gonna be hard for an LLM to do that since that context lives outside the code. But maybe it would, if you hook it to e.g. a ticketing system and include relevant tickets so it can grab context.
For instance, in your first example, why was that change needed? It was a fix, but for what issue?
In the second message: why was that a desirable change?
I disagree. If you look back and all you see are commit messages summarizing the diff, you won't get any meaningful information.
Telling me `Changed timeout from 30s to 60s` means nothing, while `Increase timeout for slow <api name> requests` gives me an actual idea of why that was done.
Even better if you add meaningful messages to the commit body.
Take a look at commits from large repositories like the Linux kernel and we can see how good commit messages looks like.
> Interesting idea. But those say what’s in the commit. The commit diff already tells you that. The best commit messages IMO tell you why you did it and what value was delivered.
Which doesn't include what was done. Your example includes both which is fine. But not including what the commit does in the message is an antipattern imho. Everything else that is added is a bonus.
Many changes require multiple smaller changes, so this is not always possible.
For me the commit message should tell me the what/why and the diff is the how. It's great to understand if, for example, a change was intentional or a bug.
Many times when searching for the source of a bug I could not tell if the line changed was intentional or a mistake because the commit message was simply repeating what was on the diff. If you say your intention was to add something and the diff shows a subtraction, you can easily tell it was a mistake. Contrived example but I think it demonstrates my point.
This only really works if commits are meaningful though. Most people are careless and half their commits are 'fix this', 'fix again', 'wip', etc. At that point the only place that can contain useful information on the intentions are the pull requests/issues around it.
I don't know when do you think I wrote that "the how" was needed because of course it's not needed. Again, OP wrote about just having "the why" in the message which is bad imho and you need "the what" there as well. As per your commit, the title is "fix crash when enabling pass-through port" which is exactly what I mean - it says what was done in a clear manner.
Typically I put the "why" of the commit in the body unless it's a super simple change, but that's a good point. Sometimes this function does generate a commit body to go with the summary, and sometimes it doesn't. It also has a habit of only looking at the first file in a diff and basing its messages off of that, instead of considering the whole patch.
I'll tweak the prompt when I have some time today and see if I can get some more consistency out of it.
When you squash a branch you'll have 200+ lines of new code on a new feature. The diff is not a quick way to get a summary of what's happening. You should put the "what" in your commit messages.
That's actually pretty useful. This could be a big help in betting back into the groove when you leave uncommitted changes over the weekend.
A summary of changes like this might be just enough to spark your memory on what you were actually doing with the changes. I'll have to give it a shot!
Here's the script: https://github.com/nozzlegear/dotfiles/blob/master/fish-func...
And for this change [1] it generated these messages:
[1] https://github.com/nozzlegear/dotfiles/commit/0db65054524d0d...