|
the talent deficit posted:typescript is a compiler that produces javascript. what if typescript was a tool that could type check an (enriched) javascript AST? then you could also use the typescript type checker on your hobby language that compiles to javascript AST. this doesn't work because nobody wants to look at error messages about code that got generated, instead of the code they wrote. It's a considerably harder problem than you're giving it credit for. (I agree with the general idea you're going for though...) quote:consider also async/await. in c# (and probably most other implementations) async/await is just some annotations that tell the compiler to convert imperative code into state machines. in a theoretical AST first language that allowed arbitrary annotations you could build your own async/await tooling (assuming the language has the primitives necessary, of course) without needing changes to the compiler or the parser you also have the expression problem to worry about. async/await introduces new productions and new analysis (tranformations). Put multiple of those together and things go boom, unless you have some solution... so it's not just "gosh expose an ast guys" it's a fundamental problem quote:there's a million things you can do if you have direct access to an AST that is designed with extensibility in mind. the problem with applying this to current languages is that either their AST wasn't designed with this in mind (scheme, lisp, elixir) or that their AST just isn't readibly accessible. or both "extensible language framework" are the keywords you're probably looking for. imo this is maybe the coolest: http://melt.cs.umn.edu/
|
#
?
Aug 28, 2017 23:39
|
|
|
|
| # ? May 22, 2024 14:18 |
|
|
p much anything by cyrus omar too https://www.cs.cmu.edu/~comar/
|
|
#
?
Aug 28, 2017 23:42
|
|
|
yep, Omar is cool. I guess also mbeddr is worth a mention too: http://mbeddr.com/
|
|
#
?
Aug 28, 2017 23:49
|
|
|
but afaict you're still begging the question about these amazing things available if you could poke at an AST that you somehow can't do with llvm?
|
|
#
?
Aug 29, 2017 00:05
|
|
|
well, the idea i was getting at was composition. you can modify llvm in multiple different ways but then if you want to put them together, that's a "good luck have fun lol" task to an experienced compiler engineer, never mind regular programmers
|
|
#
?
Aug 29, 2017 00:07
|
|
|
Do you just want Roslyn or something?
|
|
#
?
Aug 29, 2017 00:09
|
|
|
havelock posted:Do you just want Roslyn or something? truffle/graal is probably a better example, but not tied to a single vm
|
|
#
?
Aug 29, 2017 00:23
|
|
|
crazypenguin posted:this doesn't work because nobody wants to look at error messages about code that got generated, instead of the code they wrote. It's a considerably harder problem than you're giving it credit for. (I agree with the general idea you're going for though...) i don't think it would be easy or even necessarily better than what we have now. i'd just like to see a mainstream language attempt it to see how it goes
|
|
#
?
Aug 29, 2017 00:31
|
|
|
the talent deficit posted:right, that's why scheme is not really a great example. you can't really build a type system as a macro
|
|
#
?
Aug 29, 2017 02:03
|
|
|
the talent deficit posted:i don't think it would be easy or even necessarily better than what we have now. i'd just like to see a mainstream language attempt it to see how it goes have you considered that multiple grad students have likely attempted this project, and none of their languages have become mainstream? perhaps this is because most people care only about their compiler's input and output, and they don't give two shits about the intermediate representations used in the optimization pipeline? if you really want this feature, maybe you are taking code generation a little too far?
|
|
#
?
Aug 29, 2017 02:04
|
|
|
as a specific example, you can absolutely do async/await with code transformation ultimately, what is a compiler but code transformation? but you're probably going to get really really poo poo results if you do
|
|
#
?
Aug 29, 2017 02:19
|
|
|
just to make sure i'm not misunderstanding here, what is it exactly you're looking for? 1. a program that takes as input another program, runs it through the parsing stage of the compiler to produce an AST, then does some transformation on this AST? this is fine and good, the easiest way to get started is with LLVM, it is really not that hard to add a transformation to the pipeline 2. a program that pre-processes another input program, replacing custom syntax elements with standard syntax, so they can be processed by the standard toolchain? and it does this by presenting to you a truly abstract AST that supports extending the syntax of the language? this is a nasty thing to do to readers of your program. camlp4 for ocaml is one such system, but it is impossible to understand, and even the phds who came up with it have moved on 3. a program that has access to its own AST at runtime, also extensible like in (2), allowing programmers to build out a crazy macro system that manipulates the program in a way that cannot even necessarily be macro-expanded into the concrete syntax of the base language? this is perhaps the truest expression of pure evil that has ever crossed my mind
|
|
#
?
Aug 29, 2017 03:12
|
|
|
i think OP might be using the term AST where they should be using something else
|
|
#
?
Aug 29, 2017 03:30
|
|
|
i don't know how much clearer i can be. i want to see an AST designed for accessibility and extensibility, not coupled to any particular syntax or compiler something like llvm ir but at a higher level of abstraction and not tied to any particular codegen. truffle/graal is close to what i am talking about but it's still designed as a shared low (ish) level input to a JIT/interpreter at a practical level, i guess what i'd like to see is a language that goes from textual representation (syntax) to an AST that can then be used as input to arbitrary functions that can return transformed AST or compile/execute the program. if you like you can think of this as adding arbitrary codegen passes or macro expansion to the compiler but compilation/execution aren't the only time these are interesting. consider an IDE that can continually run linters, type checkers and other tooling passes as you code. or a step through debugger that can be started from any arbitrary entry point with any arbitrary arguments. or a tool that can fuzz or generate property checks for any function on demand. or a type checker that can check only subsets of your program. or a taint checker that can detect impure functions. i realize most of these are possible today but the amount of effort required to do them is often so high that in practice they don't actually exist from a users perspective the language would look just like any other language but would probably have a lot of optional syntax elements like type declarations and annotations: code:code:and again, i realize there are a ton of obstacles with this approach. i'm not putting it forward as some panacea for all programmings problems. i'm not even saying it's a good idea. i just think it's an underexplored area
|
|
#
?
Aug 29, 2017 03:41
|
|
|
JewKiller 3000 posted:just to make sure i'm not misunderstanding here, what is it exactly you're looking for? quote:1. a program that takes as input another program, runs it through the parsing stage of the compiler to produce an AST, then does some transformation on this AST? this is fine and good, the easiest way to get started is with LLVM, it is really not that hard to add a transformation to the pipeline this is a start, but see above quote:2. a program that pre-processes another input program, replacing custom syntax elements with standard syntax, so they can be processed by the standard toolchain? and it does this by presenting to you a truly abstract AST that supports extending the syntax of the language? this is a nasty thing to do to readers of your program. camlp4 for ocaml is one such system, but it is impossible to understand, and even the phds who came up with it have moved on this is closer, but the language would be specified at the level of the AST. you could have custom languages that produce standard AST and maybe even pretty printers that can convert AST back to some language, but there's no canonical syntax. just AST. quote:3. a program that has access to its own AST at runtime, also extensible like in (2), allowing programmers to build out a crazy macro system that manipulates the program in a way that cannot even necessarily be macro-expanded into the concrete syntax of the base language? this is perhaps the truest expression of pure evil that has ever crossed my mind i don't see why you couldn't build an interpreter that can manipulate AST at runtime, but no, i want the AST to be it's own distinct thing, not just something you can modify via a macro at compile time or via weird reflection magic at runtime. there's already examples of those in the wild (reflection on the jvm is kind of close, elixirs macro system is really close) and they suck
|
|
#
?
Aug 29, 2017 03:46
|
|
|
the talent deficit posted:something like llvm ir but at a higher level of abstraction and not tied to any particular codegen. truffle/graal is close to what i am talking about but it's still designed as a shared low (ish) level input to a JIT/interpreter right, so you just want "this obscure research project, except way better. also, actually usable" good luck friend
|
|
#
?
Aug 29, 2017 03:48
|
|
|
the talent deficit posted:this is closer, but the language would be specified at the level of the AST. you could have custom languages that produce standard AST and maybe even pretty printers that can convert AST back to some language, but there's no canonical syntax. just AST. so how do i read the program? i mean *i* can read llvm ir but i'm pretty sure that is not a widely held skill. how much higher level can you get without assuming particular language features?
|
|
#
?
Aug 29, 2017 03:48
|
|
|
JewKiller 3000 posted:so how do i read the program? you write a function from some syntax to AST (or use someone else's). no one* works directly in the AST * some idiots will work directly in the AST
|
|
#
?
Aug 29, 2017 03:53
|
|
|
Lutha Mahtin posted:right, so you just want "this obscure research project, except way better. also, actually usable" good luck friend i think i was pretty upfront about that, yeah
|
|
#
?
Aug 29, 2017 03:54
|
|
|
the talent deficit posted:you write a function from some syntax to AST (or use someone else's). no one* works directly in the AST that's how i write my program, but how do i read your program, which quite possibly uses language features that cannot be (efficiently) expressed in my concrete syntax of choice? ultimately it sounds like everyone will have to work in or at least understand the AST level, and i think that may be too much to ask of programmers who were not enamored with their compilers class
|
|
#
?
Aug 29, 2017 03:58
|
|
|
JewKiller 3000 posted:that's how i write my program, but how do i read your program, which quite possibly uses language features that cannot be (efficiently) expressed in my concrete syntax of choice? ultimately it sounds like everyone will have to work in or at least understand the AST level, and i think that may be too much to ask of programmers who were not enamored with their compilers class you either write an AST -> your syntax pretty printer or you use whichever syntax i chose, i guess
|
|
#
?
Aug 29, 2017 04:06
|
|
|
I don't get what the problem we're solving is. So you've got some ability to represent an AST. In various stages of annotation, perhaps. Now what?
|
|
#
?
Aug 29, 2017 04:29
|
|
|
just use m expressions
|
|
#
?
Aug 29, 2017 05:59
|
|
|
fart simpson posted:just use m expressions m'expression
|
|
#
?
Aug 29, 2017 06:34
|
|
|
VikingofRock posted:m'expression tips paren
|
|
#
?
Aug 29, 2017 06:39
|
|
|
i mean, there are plenty of languages that allow exactly that kind of static manipulation of programs, with most languages in the lisp family being an obvious example; i am not an expert in using them but i feel confident there's some way to emit static diagnostics from a lisp macro. if you want that + a normal type system i'm sure there are lisp-like languages that does that, the primary societal purpose of lisp is to safely occupy lisp hackers by presenting them with an endless sequence of obscure lisp feature interactions to implement. if you want this in a source syntax that's not literally already a tree in prefix notation then that's more challenging because your dsl request probably means literally adding new grammatical productions to the language, which i know you can do in agda but i don't know if agda has a macro system to go with it. there are also things like template haskell but that's quite a bit less powerful than you're talking about. if you want to do a cfg-based analysis then i mean that's literally writing a compiler pass and you should probably instead look at languages with specific features designed to directly support common use cases for such things like linear/affine type systems or maybe session types
|
|
#
?
Aug 29, 2017 07:56
|
|
|
sarehu posted:I don't get what the problem we're solving is. So you've got some ability to represent an AST. In various stages of annotation, perhaps. Now what? copying obnoxiously unreadable Haskell code from a PL blog and pasting it to your IDE which turns it into automatically understandable Java
|
|
#
?
Aug 29, 2017 09:10
|
|
|
the talent deficit posted:taint checker
|
|
#
?
Aug 29, 2017 09:44
|
|
|
Max Facetime posted:copying obnoxiously unreadable Haskell code from a PL blog and pasting it to your IDE which turns it into automatically understandable Java intellij does this with java and kotlin you can paste in java and it'll auto-convert it to kotlin or vice versa. it's pretty sick e: that said, doing this with languages with different standard libraries and poo poo aint an ast problem. like, i think you could do this with every language that runs on the jvm but haskell->java is a whole different problem
|
|
#
?
Aug 29, 2017 16:39
|
|
|
the problem with your thinking is that you think that typescript already operates mostly on an ast. an ast is only one intermediate representation of your code, and honestly, it's not even a good one. from an ast you have to do control and data flow analysis. that's two more transformations of the ast to a new representation. the hard part of the language is rarely the parser (unless you're compiling c++). you autogenerate that, maybe add some error messages, and move on.
|
|
#
?
Aug 29, 2017 16:48
|
|
|
Arcsech posted:intellij does this with java and kotlin it'll also translate to Scala
|
|
#
?
Aug 29, 2017 19:18
|
|
|
oh good, some random dude is going to add integral generic arguments to swift, this ought to turn out well
|
|
#
?
Aug 29, 2017 23:52
|
|
|
rjmccall posted:oh good, some random dude is going to add integral generic arguments to swift, this ought to turn out well So, as someone who is knows very little about this, what's wrong with integral generic arguments? They seem pretty useful to me in C++, for example.
|
|
#
?
Aug 30, 2017 00:10
|
|
|
its a huge and complicated feature and this dude has no idea how to do any of it
|
|
#
?
Aug 30, 2017 00:15
|
|
|
rjmccall posted:its a huge and complicated feature and this dude has no idea how to do any of it whats so complicated about it? (honest question)
|
|
#
?
Aug 30, 2017 01:03
|
|
|
Shinku ABOOKEN posted:whats so complicated about it? (honest question) not super familiar with swift specifically, but my questions for an implementation would be: what operations are allowed on integral type vars? are there any constraints? how do you denote an integral argument versus a type? can integral arguments be inferred?
|
|
#
?
Aug 30, 2017 01:37
|
|
|
i forget do you still work on swift or not
|
|
#
?
Aug 30, 2017 06:32
|
|
|
what's an integral generic argument? i tried googling.
DONT THREAD ON ME fucked around with this message at 15:33 on Aug 30, 2017 |
|
#
?
Aug 30, 2017 15:31
|
|
|
a generic argument that's a (numeric) value instead of a type. like say, you have a dynamic list of poops and it's list<poop>, but a statically sized array of poops is array<poop, 420>: first generic argument is a type (the type of array elements), the second is an integral argument (the number of array elements)
|
|
#
?
Aug 30, 2017 16:09
|
|
|
|
| # ? May 22, 2024 14:18 |
|
|
i think its like Array<3> so you know it only has 3 elements or Matrix<3,5> etc and the compiler can error-check/optimize based on that e:  i guessed it right
|
|
#
?
Aug 30, 2017 16:10
|
|
