I think this project is great, but I tend to agree with others - it's going to be somewhat doomed by second-system effects and the fact that coding is a social art form. You have to get other developers to buy in to what you're doing to be successful.
So what I wonder is this: is there a migration path instead? If the goal is to ultimately work on ASTs instead of treating software as a giant string, is there a way we can make incremental transformations to languages to move in that direction.
So far what I've considered is this:
1 - Automated formatting such as gofmt which eliminates any decisions around whitespace to culture people towards treating code as data rather than prose
2 - Language-aware editors that constrain user input to valid states based on 1
3 - Plugins for version control systems to optionally do diffs on ASTs rather than plain files
4 - Compilers that can operate on ASTs instead of strings
5 - Checking in ASTs rather than strings based on 3 and 4
6 - Updating editors 2 to operate on ASTs since 5 makes it reasonable to always work on ASTs rather than text
7 - Creating new styles of code editors that visualize and manipulate ASTs in ways that are independent of their text representations
There's probably steps I'm missing. I'm only just getting in to this space so I'm ignorant of most research that has been done in this area.
I think the closest thing to a migration path is the Racket programming language.
When you get right down to it, "working on ASTs" basically means working in lisp. The idea is you have a canonical, external format for the AST; it might as well be s-expressions. The trouble is that lisp culture is highly resistant to any other syntax than raw s-expressions. They say things like "making alternate syntaxes is a rite of passage, but everyone gets used to the parentheses in the end."[1]
Racket alone encourages the creation of alternate syntaxes, and provides the tools to make it easy. You can have any syntax you want, and yet never be hamstrung by syntax decisions because you can always make a new version of your #lang, and it'll all be nicely compatible because it gets turned into s-expressions anyway. This isn't just theoretical - you can for example already write Python in Racket [2], except with full access to all of Racket's libraries as well as Python's. This turns the social aspect to its advantage. Imagine if Python had been written in Racket to begin with - the 2->3 nightmare would have been a non-issue.
I hope that eventually, all languages will have swappable syntaxes with a canonical s-expression form, and be distinguished purely by their fundamentally incompatible semantics. We may find we need far fewer incompatible silos that way.
Incidentally, and I don't think it's a coincidence, Racket's IDE is the furthest along the non-textual route of any "real" language I've seen. It's very next-gen - you can paste an image file directly into source code, for instance (try it, (define my-image <pasted image>) actually works!).
[1] I view all "rites of passage" with suspicion. Like the "rite of passage" of wielding 'rm' incautiously, they usually indicate a human factors failure.
Lisp has various parser for language implementations. There is even a relatively complete Python implementation in Lisp.
Having Python syntax doesn't give you access to all the Python libs. Lot's of things need the Python runtime and also its support for non-Python extensions.
Literally hundreds of languages had been implemented on top of Lisp over the years. My Symbolics Lisp Machine had already in the 80s full C, ISO Pascal, Fortran, Ada and Prolog implementations written in Lisp. I was running on my Lisp Machine the original X11 server written in C and TeX written in Pascal...
Python is also not an alternative syntax on top of Racket or Lisp ... it's a different language.
I'm completely ignorant on this, but I'm wondering if there could be a strict superset of haskell or other pure functional language that is inherently visual. It seems this is what lambdu is, but it would definitely benefit the language to have it compile down to some readable functional code whenever needed.
Lambdu, Unison, etc. are all great in principle but share the same fatal flaw: no layering.
It's just one monolithic component trying to take on type systems, effect systems, syntax, version control, compiling, editing, distributed execution, etc.
All of these problems taken decades just to get to the shitty everything-is-text/worse-is-better state we're in now. I'm skeptical that any single team can solve them all at once, even with the most radical vision and competence.
Part of the reason we're in the shitty everything-is-text/worse-is-better state we're in now is because we haven't invested a lot of effort in the top-down design of systems that work really well.
This is in contrast to the current state of development, which was driven by Moore's law and the rapid improvement of single-core processors to consist mostly of imperative programming. As soon as we started hitting a wall in single-threaded CPU improvements and moving to multi-thread and multi-cores, the imperative programming model began to rapidly show its limitations. After all, Von Neumann meant for his architecture to be a prototype, and we went ahead and built an entire industry on top of it.
Leaving everything in the development ecosystem up to market forces has left us in a state that is entirely too complex. There is far too much accidental complexity in developing even the simplest applications today.
If we purposefully design smarter systems, maybe we can cut out layers out of the stack entirely, so worrying about things like distributed execution is a thing of the past for 99% of programmers.
> we purposefully design smarter systems, maybe we can cut out layers out of the stack entirely, so worrying about things like distributed execution is a thing of the past for 99% of programmers.
Actually I disagree. If your layers represent conceptual abstractions, and not arbitrary divisions of your code stack leaking details from below, then _more_ layers is actually a good thing.
I think these kinds of projects are mostly "conceptual" rather than something that will straight up take over the world, providing inspiration for people writing those things that will actually get used. I feel like it is extremely important that this kind of work gets done, and I think people should also be more eager to actually try out and really use all sorts of novel systems.
A lot of these things are solved, or at least there is a tremendous amount of experience to draw from now. It's more a problem of packaging the different pieces together into a cohesive system.
It all sounds nice, especially considering that the expansion of the code-literate demographic should eventually necessitate more intuitive tools than the languages & environments we use today, but it also seems intimidatingly different and severely under-supported. Overall, I feel like I can confidently ignore this project because it appears doomed to fail and/or ahead of its time.
So what I wonder is this: is there a migration path instead? If the goal is to ultimately work on ASTs instead of treating software as a giant string, is there a way we can make incremental transformations to languages to move in that direction.
So far what I've considered is this:
1 - Automated formatting such as gofmt which eliminates any decisions around whitespace to culture people towards treating code as data rather than prose
2 - Language-aware editors that constrain user input to valid states based on 1
3 - Plugins for version control systems to optionally do diffs on ASTs rather than plain files
4 - Compilers that can operate on ASTs instead of strings
5 - Checking in ASTs rather than strings based on 3 and 4
6 - Updating editors 2 to operate on ASTs since 5 makes it reasonable to always work on ASTs rather than text
7 - Creating new styles of code editors that visualize and manipulate ASTs in ways that are independent of their text representations
There's probably steps I'm missing. I'm only just getting in to this space so I'm ignorant of most research that has been done in this area.