I've done some elisp (and a tiny bit of racket) and some Clojure and honestly I struggle to see why I would ever pick a racket/scheme/elisp over Clojure unless I desperately needed to do a ton of C ffi. Clojure is much more opinionated but I never find myself feeling like it's constraining or a straightjacket
When do people say to themselves "screw this, I need a more flexible tool like Racket"? Is it when you get super deep in the macro magic? (I'm not reallt sure how the Clojure macro system compares to the Scheme one)
I've done way more Clojure than any other Lisp, and I'm now transitioning to Racket. Primarily, because I want to get away from the JVM. One thing about Clojure that is not easy to put into words, and that I miss, is just how "smooth" working on it is. The Clojurisms, as they come to be known, are very well thought out, and the whole language fits together like a perfect puzzle, or at least in my experience. To come back to the JVM thing, I find that Clojure is applicable only serverside, where you can have the slow starting hundreds-of-megabytes JVM running non-stop, or in the browser, through ClojureScript, and it's really perfect for those environments. Though as I shifted more towards non-server environments, and I need to use FFIs, Racket is the right blend of expression and performance.
Yeah, I'm in the same boat. I mainly used Clojure, but have been doing more Racket lately. Mainly for short scripts, because it doesn't have the warm-up time of Clojure. Racket is a beautiful language, but some things in Clojure are just opinionated in a very nice way.
For example, the seq abstraction and the fact that most sequences and functions are lazy by default is really nice. It also makes more use of polymorphism, so you can e.g. call map on a hash-map, a vector or a lazy seq. Also the fact that maps and keywords act like functions is extremely convenient. Similarly, let is actually let* and binding forms allow destructuring. Racket has all this, but you have to remember to use e.g. match.
Racket macros are a work of art and I'm only just exploring them. However, the surface area is very large. In Clojure once you know the language and some of the standard library you can write macros. In Racket it seems there's a whole other language you have to learn. It's clear though that they're extremely powerful, so I probably just need to invest more time in it.
All in all my ideal language would be Racket with some of the conveniences and choices of Clojure. Gerbil is another lisp that looks interesting; maybe I'd steal some stuff from there too. Of course, Racket is extremely well suited to building languages, so I could actually build this ideal language if I wanted (and had the time and skill required).
While being a general purpose programming language, Racket is intended for Language Oriented Programming[1]. I was looking at Lisp Game Jam 2018's submissions when i stumbled on this game https://euhmeuh.itch.io/lisp-vs-blub-empire. The author made a throwaway language by wrapping Web Assembly intermediate language into racket using less than 250 lines of code[2]. This is what Racket is really about; an environment to quickly create a specific programming language for a specific problem[3].
I just find racket to be more fun. If I'm blowing off some steam on the weekend, racket is what I reach for. It feels clean like scheme, but has the "batteries included" feeling of python. Clojure is alright, but it feels more like work than fun. I don't really know how to justify that feeling, but that's how it feels. I guess it's little things like racket/match being an 'included battery', and having a ton of useful forms already defined out of the box (e.g. match-lambda and match-letrec-values.) Clojure's core.match is pretty spartan in comparison, last I checked.
As far as macros go, I greatly prefer racket's syntax-parse to anything else I've tried.
I think I was trying to articulate the same thing before, why Racket instead of more-practical-tool-here for "experiments/fun" and it really is just because it's more fun to work with, nearly everything you need is in there!
Also personally something I wish more functional languages would have is a little bit of Racket's maximalism with its forms like: (for/list ([e (range 10)]) e) [1] and for/fold, including the nested ones and the like that make mapping over things and even filtering a dream!
I implemented racket's for loops for guile scheme (my server is down since a week, so my hosted documentation isn't up, but you can find it in the repo linked below). It is maybe not a trivial macro, but I didn't need to struggle much to reach about 90% feature parity with racket.
The code it outputs is almost always as fast as a hand-rolled named-let (just as with racket), and you can port it without much difficulty to other schemes. I support most of the sequence iterators (in-range, in-list etc) and support non-tco loops to create things like lazy streams.
The flexibility might not just be in the tools themselves, but in the culture of the language. Elixir for example has really powerful macros, but one of the main books about metaprogramming says "the first rule of macros: don't write macros". So you have some beautifully designed DSLs like Ecto.Query showing how powerful they are, but you rarely go trying to solve your problem by using that power for creating your own DSL or language (and mostly just use 'use' macros for code injection in your module).
So even if one language's power is enough for someone's programming ambitions, they might still want to go for an environment with more like-minded people that are more supportive of using (and in the perspective of some, abusing) that power.
I find the usability and elegance of Racket to be far better than Clojure. Clojure feels messy, mostly due to the JVM. Racket feels clean and fun and expressive. The ecosystem is better integrated as well.
Clojure's performance is of course better, though.
I've done a bit of both, and while I probably prefer Clojure as a language, the Racket docs are more comprehensive. That made Racket more accessible for me.
Also, trying to figure out how to build (non-web) GUIs with Clojure and Swing wasn't a pleasant experience for me.
When do people say to themselves "screw this, I need a more flexible tool like Racket"? Is it when you get super deep in the macro magic? (I'm not reallt sure how the Clojure macro system compares to the Scheme one)