At Fig (https://withfig.com), we had to write a bash parser from scratch. These projects all work well in the IDE context, but in a repl environment (ie when typing in your terminal) they don't support incomplete commands very well. e.g if I type `echo "$(` the above parsers will break as the double quote and parentheses are not closed.
Cool! Some of the biggest problems we have seen are:
* scripts being poorly documented: what's the point of having a script if no one knows how to use it
* scripts going stale: often built by one person / an infra team. When they break, no one knows how to fix them
* creating scripts: bash isn't the most popular language these days, but it's still supported just about everywhere. How can we make building them easier and faster but still just as powerful
Tree-sitter is more general: it aims to provide support for many languages and so its bash support is lacking a little. But it's very fast. It's designed for IDEs (hence why bash-lsp is using it). It caches its results and only re-parses on newly changed code blocks.
sh by mvdan and shellcheck are both built specifically for the shell. They both have really robust grammar. shellcheck is great for error handling. sh is great for building ASTs (it's more of a true parser)
As always, it depends on your use case. If you need a descriptive summary of errors / warnings in a shell script and performance isn't crucial, use shellcheck. If you need a really robust parser and AST output, use sh. If performance is crucial and/or you eventually want to support different languages, use tree-sitter.
To me, the main value of sh is shfmt – I use it like gofmt and never have to think about formatting again. It also integrates with GitHub's super-linter, where you can have your preferences in an .editorconfig file and then enforce them automatically.
Development tools like this with an obscene amount of dependencies, not to mention multi-language dependencies, is extremely worrying and irresponsible for the future of software development. Especially since they're not even native to bash's environment! If you would have told me 5 years ago I'd need to install node and typescript in order to get auto completion for bash, I'd have called you crazy. Yet here we are.
> irresponsible for the future of software development
Not true. This is highly interchangeable tooling - it can have whatever dependencies you want, because there's no "dependency explosion" that happens with libraries. LSP is a standardized protocol - you can swap it out with anything you like.
Moreover, people rarely build interactive tools on top of other interactive tools. Now, if this were a library with a bunch of dependencies, that would be a whole different matter - then any program or library that used this would get all of the transitive dependencies - but, it's not.
Go and reimplement this tool yourself. RIIR, make a statically-linked 500kb executable with no perceptual execution time, and eat their lunch, because your tool is a drop-in replacement.
Is a Node dependency a bad idea for this tool? Arguably yes. Is it "irresponsible for the future of software development"? That statement is so false that I suspect it's being used to try to emotionally manipulate readers instead of making a coherent argument.
Also...are you aware that Bash already tab-completion, even if it's not automatic?
I'm beginning to wonder if this is a very subtle April Fool's post...
I like minimalism, I agree with cutting down dependencies, but the person who does the work gets to pick the language. If I did it, I'd do it in C, Rust, or Kotlin, but I appreciate having an LSP for a new language nonetheless. Looks like it has no runtime library dependencies, then that removes risk of supply chain issues, so I think they've done a good job.
> node and typescript in order to get auto completion for bash, I'd have called you crazy
It's worse than you think.
This uses something called Tree-sitter: some project written in C for providing parsers for numerous languages.
The Tree-sitter parser for Bash itself somehow requires Typescript, though the parser is in C.
This Bash-LSP project uses a binary blob of that parser: it is C compiled as C++ into WebAssembly.
Bash-LSP, as well as the Tree-sitter Bash parser, can basically be called IDE tooling written by and for TypeScript programmers, who sometimes have to mess with Bash scripts.
The target audience doesn't mind TypeScript and Node dependencies, and in fact likely finds that preferrable to any other kind of dependency, because working with TypeScript dependencies is fresh in their minds, requiring little ramp-up to handle another one.
> The Tree-sitter parser for Bash itself somehow requires Typescript, though the parser is in C.
You can use tree-sitter to produce nothing more than C parser file that only depends on a very small tree-sitter C library.
So you can use tree-sitter to produce a parser that does not require Typescript.
However, there is whole other side to tree-sitter which is about filtering/manipulating/searching the AST results produced by that C parser and this layer has binding to many languages.
I suspect this Bash LSP project is using the tree-sitter Typescript bindings for their AST processing.
> The Tree-sitter parser for Bash itself somehow requires Typescript, though the parser is in C.
This is not necessarily true if you can write a compiler for a tree-sitter grammar that outputs the parser code in your favorite language. I know semgrep does this to make parsers for OCaml. But, of course, that's more work to be done.
And either way, you don't need TypeScript to run this language server. You just need node. If you're not happy with that, well, consider this repo a reference and get cracking.
You can always tell who hasn’t used a tool because they leave comments like this. Same thing with complaints about node_modules. Anyone who’s actually worked with frontend tooling knows it’s light years ahead of the tooling in most backend languages.
The developer experience out of the box with TypeScript + VSCode + Next.js is 1000x better than any stack I’ve ever worked with in Python, for example. So what if you need to pull in some dependencies? At least I don’t need to figure out what the best way to setup a Python environment is this month.
(In fact, the current Python package manager du juor, Poetry, took most of its ideas from what Yarn implemented five years ago.)
And what’s this about being irresponsible? You can easily pin your dependencies, use a lockfile, and even bake the cache into your Docker images. There’s no reason you can’t have a program that will run the same in two years as it does now. And setting that up is a lot easier than it is with other languages.
Are you serious? About a third of my job is frontend with typescript. The rest is backend and some application dev and embedded. Frontend is seriously the worst environment. It's great if the magic works. But that's the easy part. Once something breaks, and it will, you have to descend to the seventh circle of hell to figure out where something in this mount everest of tooling and dependencies broke.
I love the idea of frontend dev. But the tooling takes all of the fun out of it.
But I don't think your problems with FE Dev/TypeScript are "TypeScript" problems so much as they're problems around the tooling you're using with `node`.
And I agree. Unless it works out of the box, it's a half-day of profanity at least.
But writing a command-line tool in TypeScript doesn't require any of that. The comment focused on FE development with Next.js -- I have no experience with that, but a simple React-based solution using `create-react-app` works out-of-the-box for a lot of things[0]. I will say, it does sometimes feel like everything TypeScript gets right is made up for by everything "all of the bundlers/tooling around it" screws up, but it's gotten monumentally better. You're always going to be weighing pain points against eachother, and folks that do front-end web development all day long are going to find the problems they routinely encounter to be "simple", possibly even "intuitive" based on how the rest of the ecosystem works.
From my perspective -- partly on the inside, partly on the outside -- I seem to have to muck around with the equivalent of "build scripts" about as often as I used to (not) enjoy in the C/C++-land. But dammit-all if "time to completely working" in TypeScript isn't always twice as quick. TypeScript, configured correctly, catches so many run-time issues at compile-time[1]. For me, the distance between "it runs" and "it runs correctly" is often shorter in TypeScript despite it not being my primary language.
[0] To be sure, it's hard for me to say `create-react-app` without the obligatory `fscking` inserted... it's got many problems, but at least half the time it's an implementation detail.
> Anyone who’s actually worked with frontend tooling knows it’s light years ahead of the tooling in most backend languages.
The developer experience out of the box with TypeScript + VSCode + Next.js is 1000x better than any stack
You should practice your April fools jokes to be less obvious, that post is not gonna make it.
I don't know that I'd agree with all of those statements, but my expertise lies in C#, with a bit of TypeScript/JavaScript/Front-end Web thrown in from time to time. I used to write a lot in C++, have dabbled in C...
The reason I don't write everything in TypeScript has nothing to do with TypeScript. Language-wise, it is my favorite experience, hands down. I can express things in a type-safe, strict manner, and do things generically that simply cannot be done outside of dynamic languages, while still gaining the benefits of a statically typed language.
Not taking node into account[0], TypeScript on its own is my favorite language and it's not the one I write software in every day. While I agree with your arguments around pinning and such, node is a bit of a victim of its own success and suffers from dependency-hell in a way that is truly unique and interesting all on its own. The only way to be certain that code is good is if you write it, barring that ... a trusted, supported third-party for the important things, and inspect the rest. The problem on the node side is the impossibility of the last task, and if you aren't doing it, neither is your trusted third-party. I'm not saying it's hopeless -- I'd even wager a guess there's a comment in my past that is far more harsh on `npm` than it deserved/deserves[1].
I don't think "node-based" solutions for shell are bad. The biggest problem I tend to run into with shell scripts in constrained environments are related to dependencies. With node, `install`, done (usually). I target "what's probably there" like most people. Sometimes it's not. In the past, when this was a bigger problem than it is now for me, I would throw my hands up and build it in whatever I wanted because "if I had to get a waiver to add `jq` or some other thing to this box, I might as well have them put `python` on there" -- same amount of work arguing over BS but less work writing it. I'd probably apply that to `node` these days but those problems aren't mine any longer. :)
[0] Which some will point to `npm` as the whole reason they use node while others will point out why it's the entirety of its problems.
[1] And I hate the attitude of some that implies `npm`s problems have easy solutions, "if they just ...". No. It's almost everyone's first language. It's really easy to publish packages. There's a gajillion of them out there. (Yet, it works ... every? ... time I need to solve something using it.)
But there is a lexer and parser for Bash inside Bash. Bash could either provide the LSP protocol itself, or else open up some primitives for a script to gain access to the info somehow.
There's more languages native to unix-land than Bash, you know. Find me a linux distro that doesn't have GCC or Clang installed by default, or even Python for that matter.
Bash is written in C so depending on C doesn't add a dependency.
Python is also the 2nd most popular scripting language in the Linux world; it's even part of the Freedesktop base runtime. Though I don't know if it's standard in BSDs.
What are you talking about? TypeScript has the best Language Server libraries, if you want maximum productivity when making a Language Server, you run with TypeScript. I don't even like TypeScript but I can at least acknowledge that it can make sense to implement a Language Server in it.
Personally I would have used Go for this exact reason, but it's reasonable to use TypeScript too.
> Development tools like this with an obscene amount of dependencies, not to mention multi-language dependencies, is extremely worrying and irresponsible for the future of software development
So you want to tell, it's better to have no tool at all (given pre-tree-sitter Bash LSP doesn't exist), than to have a tool with a lot of dependencies? I get that some may avoid having huge number of dependencies on production, but when it comes to local development tools, I couldn't care less. It's 2021, I can buy 1 TB SSD for less than 100 bucks.
You said it, sarcastically, but that didn't mean I didn't think it. At first blush it sounds a little crazy, but the more I think about it...
Consider my problem: I use `zsh` and have created, collected and organized a large number of completions which I synchronize among several workstations. It frustrates me to no end that I cannot take advantage of these completions in very many other convenient places, like VSCode[0]. The benefit of writing it in `zsh` (I can't speak for bash) is that seems like it'd be pretty simple to wire up the LSB as a sort of "completions proxy" that simply takes the inputs, passes them to a completion script, parses the output and formats it in the appropriate JSON response. The utility of being able to have a single script (or set of scripts) that I can include in my shared profile directory which will run and handle a `zsh` LSB implementation on-demand (i.e. doesn't need installing, sits in my scripts directory and runs everywhere) would be a huge feature.
This will speak to my ignorance, but can you host a web server (in a manner that can be consumed) from just bash and a lean environment[1]? Could you host an LSP without one (i.e. is there enough to use a socket and would it be enough to point it at the right place/scheme?). I believe there are `zsh` extensions which can facilitate some/all of this, but I'm not sure if it would work out of the box. The downside to developing it in `zsh`, other than the language itself[2] is performance. Both of those, assuming "the hosting/serving" can be handled easily, don't represent big problems, maybe they are. But performance and the language should be up to what remains outside of that.
[0] I had `nvim` set up correctly, once to work with completion scripts and was able to translate that into VSCode doing the same, about two years ago. I'm still not fully convinced I didn't just dream this up because the few times I've looked into it, since, I've gotten nowhere (admittedly, I've put little effort into researching).
[1] I'm thinking 'busybox', minimal gnu-related things, ideally requiring as little as possible beyond a relatively recent version of `bash` so that "if it's got bash, I can use it".
[2] I use `zsh` because it's easier for me to write things in it than `bash`, and at times it can be fun. But there's a reason we don't build solutions using it.
I have been using this for a year or so and it is awesome. I use with neovim + LanguageClient-neovim. I don't really understand why people are complaining about the dependencies - on macos just brew install node@14 and npm install -g bash-language-server. Check for updates occasionally via npm outdated -g. I really have no idea what the dependencies are other than node and nor do I care - its just another app on my system.
I clicked through to the point of skimming the documentation for tree-sitter before giving up on understanding this quickly. There are too many components that I haven’t seen for me to extrapolate out and get to what this does in general terms.
Could anyone with more knowledge please explain what this is and how one might utilize it? Fwiw, I’m deeply familiar with Bash; I just don’t recognize the rest.
Language Servers are pieces of software that implement IDE-like features (like go to definition, find usages, call-hierarchy, rename method, etc) for a given language in an editor-agnostic way.
With that, any editor that supports Language Servers can use those to provide better support to a language without implementing any details.
It doesn't seem editor agnostic to me. It seems geared towards vscode.
The idea is also not new - python has had docstrings since the 90s. They never thought to show the full definition on hover or autocomplete, probably because it's noise most of the time, but a lot of people seem to like it.
Language Server Protocol (LSP) is a standard protocol for "plain old" editors (clients) to make RPCs to local processes that understand the syntax, references, and types of a particular programming language (servers). This reduces the work to support M servers and N clients to M+N, whereas the traditional monolithic approach required M*N integrations. This particular project adds a server for Bash support.
The other comments explained what LSP is - I'll explain what tree-sitter is. It's a clever parser that is very fast when it comes to reparsing a slightly changed file (it's 'incremental'). It's also very good at handling errors and continuing to parse your file.
You write a grammar in json and it gets compiled to a shared library via C.
That's simply not true; there are LSP client implementations/integrations for vim, neovim, emacs, sublime text, web environments like gitpod.io, Eclipse IDE (via Theia IDE), Jetbrains supports it in their IDEs, and Visual Studio even has some level of support for it.
LSP is _the_ way to write IDE integrations in the modern era.
(I maintain LSP tooling for F# and actively support at least half of the editors above)
It's neither true nor false, it's an opinion. Sure "LSP" has support built in or tacked on (as plugins), mostly tacked on, to other editors. Nobody is required to assume TypeScript docs for a supposedly language-agnostic protocol, or that particular flavor of JSON-RPC, or JSON-RPC, or even JSON. And the pill I don't want to swallow, that people using LSP tend to swallow, is getting full definitions with autocomplete, or hover. Most of the time I prefer to just see completions when I autocomplete, not full definitions. It is clutter.
I'm not at all sure what you mean by 'typescript docs' in your comment. The protocol for things like tooltips/signature completion literally just says 'put arbitrary markdown here'. It doesn't proscribe the structure of the text the LSP author decides to use.
There are also many different display/view methods that more sophisticated LSP implementations can implement. In my own LSP, we have different 'views' of the signatures/types/docs for
* hovers
* signature completion
* detailed type-level embedded docs in a separate pane
The official docs for LSP express the format in TypeScript rather than plain english or something like JSON schema. https://microsoft.github.io/language-server-protocol/specifi... If it pointed to TypeScript docs to brush up on it that would be one thing but it assumes TypeScript knowledge.
I think in this case, worse is better, so I'm not gonna try to produce something better, thanks.
These editors that have supposedly embraced LSP are perhaps throwing a bone to people who want to turn them into IDEs. If people don't use the LSPs, they keep the integrity of the editor - it is an editor, not an IDE. Whereas people who set up an LSP can go nuts. I'd like to see some stats about what % of neovim users are using an LSP plugin. I'm not.
Kind of cute. Jumped around with xref and found callers for functions, but can't really imagine this to be useful unless you're as crazy as I am or also responsible to insane number of these _things_.
Not sure what qualifies as doc strings, I discovered you have to use two hashes for multiple lines. More documentation on that might be helpful then I can adjust my comments to match its expectations. Also I'm pretty sure I must have broken it or something is going on in emacs lsp as it stops doing anything.
This is built on tree-sitter, but there are a couple of other interesting shell parser projects:
At Fig (https://withfig.com), we had to write a bash parser from scratch. These projects all work well in the IDE context, but in a repl environment (ie when typing in your terminal) they don't support incomplete commands very well. e.g if I type `echo "$(` the above parsers will break as the double quote and parentheses are not closed.