So if I have a fairly unremarkable setup with LSP to give me completions, what do I get by fooling around with tree-sitter. It seems like this is more geared toward building an AST, so I'm not sure how it would present itself to the end user currently?
You are correct. tree-sitter is not in competition with lsp; lsp is project wide (different files), so will do say code completion. tree-sitter is analyzing the current buffer and applying things like highlighting, brackets etc.
lsp can do some of these tings as well, but sending the entire buffer over to the lsp server every time you want to update the buffer is an expensive operation. tree-sitter does it locally.
I think at this point it’s a new building primitive mostly aimed at major-mode authors.
That said, tree-sitter should make it possible to create paredit-like implementations for languages not LISP and other stuff like that, which IMO could turn out to be really neat.
As a change, this is quite significant, but not directly aimed at end-users.
Faster and more correct syntax highlighting is the main benefit atm, as I understand it.
In general it's for functionality that needs to understand syntax but doesn't need a full compilation-level understanding of the code, that can benefit from much faster responses than an LSP server can provide and that people may want working out of the box for many languages without having to install and configure language servers, generate compilation DBs, etc.
For easy thread reading. Imenu is a way to jump around in the file M-x imenu gives menu of different types of locations (ex functions), you select functions, then you can select the function to jump to.
