Asmjit

[Lartu]

Hey, @dvkt! I've seen you recently starred https://github.com/asmjit/asmjit. This got me thinking, isn't it there a way we could embed asmjit into LDPL, so a C++ compiler is not required to compile the transpiled C++ code from .ldpl files? 🤔

[xvxx]

I think asmjit is just for assembly, so the LDPL compiler would need to produce assembly (via asmjit's API) instead of C++. 😱

I've looked around a bit for a "lightweight" C++ compiler we could embed, but I haven't been able to find one. Possibly it's an oxymoron.

There are small C compilers we could possibly embed though, like tcc. In theory we could have LDPL emit C instead of C++, then use something like Cello to handle high level stuff.

Another option would be to include an LDPL interpreter with the compiler, either by writing one or embedding something like Lua and translating to it on the fly. Then people could have both a LDPL compiler and an interpreter, depending on their system and situation.

The best option, obviously, is to drop C++ and require a GnuCOBOL compiler instead: https://open-cobol.sourceforge.io/HTML/gnucobpg.html#Why-YOU-Should-Learn-COBOL

But overall I'm not sure that there's a way to continue generating C++ while not requiring a C++ compiler be installed. Would love to find a way though.

[Lartu]

You are right. I don't really think C code would be a problem, but I'd hate to convert all the C++ codebase to C. The interpreter way sounds more interesting, maybe something like that could be added. I don't really think LDPL is a useful language for scripting, but it could be used for such things, so maybe that'd come in handy. What do you think? Maybe an internal VM would work. LDPL had one when it was first released (check the oldest releases) but it was waaaay slower than python (and LDPL atm is eight times faster than python, so that was very darn slow).

[Lartu]

(I wouldn't jump to cobol for anything in the world, but I'll read the article you linked 😂)

[xvxx]

[xvxx]

[xvxx]

[xvxx]

Maybe an internal VM would work. LDPL had one when it was first released (check the oldest releases) but it was waaaay slower than python (and LDPL atm is eight times faster than python, so that was very darn slow).

Oh, that is awesome! I never used that version of LDPL.

I think it could be fun to do a real VM. I'm not sure how far you took the previous one, but I think we could make it pretty fast since we use native (unboxed) types and there are no real function calls, arg passing, or scope. We could also integrate a JIT like Python does, maybe even asmjit.

But I'm not sure if it would be worth all the effort though, that's your call. I personally would vote that we spend time improving the compiler and improving the language standard, but I would be happy to hack on a VM or a "script" version of LDPL too!

I think showing people that there's a good use case / niche for LDPL is a good way to grow and get more contributors, and potentially having two official implementations out of the gate confuses that. But at the same time I am all in favor of experimenting and trying new things so I think it just depends on where you want to take it. Maybe it's an advantage to say "LDPL can be compiled or interpreted!"

[Lartu]

The last picture you posted was the base for the original LDPL logo and, in fact, the LDPL name!

LDPL was chosen because it was a CAPITAL LETTER, VERY RIGID, CORPORATE PROGRAMMING LANGUAGE NAME, like that of COBOL, and the original logo was a parody to that one!

The article you linked is great, it really got me excited to give COBOL another try, haha.

Oh, that is awesome! I never used that version of LDPL.

Awesome is a very big word, it was very, very slow. The VM was very badly implemented, though.

I think it could be fun to do a real VM. I'm not sure how far you took the previous one, but I think we could make it pretty fast since we use native (unboxed) types and there are no real function calls, arg passing, or scope. We could also integrate a JIT like Python does, maybe even asmjit.

This was what I would've done if we hadn't hopped to C++ transpiling. I love the idea!

But I'm not sure if it would be worth all the effort though, that's your call. I personally would vote that we spend time improving the compiler and improving the language standard, but I would be happy to hack on a VM or a "script" version of LDPL too!

What do you mean "improving the language standard"? I think the main issue LDPL has at the moment is that (for example) you cannot SPLIT a string into a vector and then split each index of the vector into another vector, for this would imply having 2D vectors (you cannot do something like SPLIT "hello there" BY " " IN myVector and then SPLIT myVector:0 BY "l" IN otherVector:0) and having no pointers you cannot really implement data structures in LDPL. But changing all this would break compatibility with already written programs. LDPL should've had normal associative arrays and vector vectors, but I guess it's too late for that. Other than that, what do you mean?

I'm happy to do whatever you'd like to spend time on, everything you've proposed sounds great. It's your language as much it is mine, so I don't have the last word here.

I think showing people that there's a good use case / niche for LDPL is a good way to grow and get more contributors, and potentially having two official implementations out of the gate confuses that. But at the same time I am all in favor of experimenting and trying new things so I think it just depends on where you want to take it. Maybe it's an advantage to say "LDPL can be compiled or interpreted!"

This is true, but in the end if we had a very good VM we could turn LDPL into a scripting language and go with it (it would even be more portable this way I guess). Maybe what you say it's also an advantage, yes. I guess at the moment we should try to find a niche for LDPL and make it grow there.

[Lartu]

Given that LDPL is not, by any chance, a widely used language (not even close) we could maybe redefine the standard, move the version up to 4.0.0 to mark that's incompatible with previous versions and just get away with it.

[xvxx]

The last picture you posted was the base for the original LDPL logo and, in fact, the LDPL name!

Okay, now THAT is awesome! I am having a blast learning about old COBOL stuff. Really cool that that's where the logo comes from!

What do you mean "improving the language standard"?

I don't have anything specific in mind, I mostly just mean focusing effort on using the language and then adding features that seem obvious or necessary over time. Maybe I should have said "growing the language standard." I can imagine more text operations maybe, like uppercase or lowercase being useful. And if it goes into scripting territory, maybe a few statements to help work with paths.

I guess I will say that the thing I want most is local variables in some form, specifically for loops. Because it's tedious to keep defining new i and len variants in non-tiny programs, or to keep track of when you're using which variable to make sure you aren't using it twice at the same time when calling a sub-proc from another sub-proc. But I think that adds a big layer of complexity to the language and I don't really have any ideas for how to do it nicely.

I sometimes wonder if there should be modules with their own scope, something like that. Each PROCEDURE: block could be its own thing, and then you'd have to import variables explicitly from another file's DATA: section. Kinda like how external works. It wouldn't fix local variables between sub-procedures, but limiting variables to a single file would help a lot with how much you have to mentally manage, maybe. Yet I find myself not wanting to suggest major changes since I think LDPL is a thing of beauty as it exists today :)

Given that LDPL is not, by any chance, a widely used language (not even close) we could maybe redefine the standard, move the version up to 4.0.0 to mark that's incompatible with previous versions and just get away with it.

That's true, pointers and structs would definitely open up a whole new world of possibilities! Especially if they were LDPL-simple but C/C++ compatible. It could be interesting to think about what that would look like, at least.

[xvxx]

Re: VM, well, I have been working on a toy VM written in LDPL. So I do like working on that kind of stuff! Maybe if no one else starts it in a few weeks I will take a stab at getting something going there. It is nice to have programs instantly run when you're debugging or building them, then have the ability to compile them for release. So I would use an LDPL interpreter if one was included.

[Lartu]

Okay, now THAT is awesome! I am having a blast learning about old COBOL stuff. Really cool that that's where the logo comes from!

Haha, thank you!

Maybe I should have said "growing the language standard." I can imagine more text operations maybe, like uppercase or lowercase being useful. And if it goes into scripting territory, maybe a few statements to help work with paths.

Yes and yes! Imports are also something that we are lacking. Maybe we could go the python way and just allow importing documents in the same directory, that'd be nice. Also, parsing sub-procedures before parsing the whole code (two-pass) would be nice, so we can call sub-procedures in the code before they are declared. That'd be awesome too. And maybe some way to implement n-dimensional vectors. I was thinking that maybe we could add a way to "prefix" vectors. This is quite weird, it would work like this:
PREFIX "_" TO VECTOR myVector
And, from now on, every time you do something like STORE 2 in myVector:0 it won't be stored in index 0 but in index _0. What would this be useful for? Well, you could do something like:

PREFIX "1:" TO VECTOR myVector
SPLIT text BY " " IN myVector
PREFIX "2:" TO VECTOR myVector
SPLIT otherText BY " " IN myVector

Ant the all the indices belonging to the first text would be in 1:X and all the indices belonging to the second text would be in 2:X, so this would be a rough way to implement - very easily - n-dimensional arrays. What do you think? Too dirty?

I guess I will say that the thing I want most is local variables in some form, specifically for loops. Because it's tedious to keep defining new i and len variants in non-tiny programs, or to keep track of when you're using which variable to make sure you aren't using it twice at the same time when calling a sub-proc from another sub-proc. But I think that adds a big layer of complexity to the language and I don't really have any ideas for how to do it nicely.

I don't think that is too much of a problem. We could just rely on C++ and allow variable declarations outside the DATA section, ditching both de DATA and PROCEDURE sections. Thus, thanks to C++ magic, variables declared in each place would belong to the scope that place has in C++. Maybe we could change the DATA section to GLOBAL DATA section and make those variables global. Would be interesting to see what happens.

I sometimes wonder if there should be modules with their own scope, something like that. Each PROCEDURE: block could be its own thing, and then you'd have to import variables explicitly from another file's DATA: section. Kinda like how external works. It wouldn't fix local variables between sub-procedures, but limiting variables to a single file would help a lot with how much you have to mentally manage, maybe. Yet I find myself not wanting to suggest major changes since I think LDPL is a thing of beauty as it exists today :)

Haha, thank you for loving LDPL the way it is. You are right, but this (but limiting variables to a single file would help a lot) would add a whole layer of complexity to the compiler I guess. But this can maybe be fixed with what you said about scoping. Another way would be to prefix the name of each variable and sub-procedure with the name of the file they belong to, or some namespace name defined at the beginning of each code (MODULE NAME "X") so if you wanted to call or use a variable from other file you'd have to prefix it's name with something like $"X"$variableName, I don't know.

What do you think about all this? I do really think great things could come from everything we've said here, making the language more versatile and useful. I really look forward to hearing what you have to say.

That's true, pointers and structs would definitely open up a whole new world of possibilities! Especially if they were LDPL-simple but C/C++ compatible. It could be interesting to think about what that would look like, at least.

This would add some complexity to the language, but yes, I agree that it would be something interesting to experiment with!

Re: VM, well, I have been working on a toy VM written in LDPL. So I do like working on that kind of stuff! Maybe if no one else starts it in a few weeks I will take a stab at getting something going there. It is nice to have programs instantly run when you're debugging or building them, then have the ability to compile them for release. So I would use an LDPL interpreter if one was included.

I would love to see that! Very interesting stuff could also come from this. Also using LDPL to compile it's own VM would be something awesome, haha!

[xvxx]

...all the indices belonging to the first text would be in 1:X and all the indices belonging to the second text would be in 2:X, so this would be a rough way to implement - very easily - n-dimensional arrays. What do you think? Too dirty?

Very interesting! I would use this in Dino, I have a few vectors that are n-dimensional using prefixes: https://github.com/dvkt/dino/blob/22db24d4d762634c6adb3ec989dc4a7d7b3d0f0d/src/vm/cpu.ldpl#L170

I guess the other approach would be:

DATA:
myVector is text vector vector 
prefix is text
PROCEDURE:
STORE "1:" IN prefix
SPLIT text BY " " IN myVector:prefix
STORE "2:" IN prefix
SPLIT otherText BY " " IN myVector:prefix

Which is nice because it doesn't add any new statements to the language, and the prefix is always explicit. But maybe we can start with the PREFIX approach and see how it works out?

I do really think great things could come from everything we've said here, making the language more versatile and useful.

Yes I like what you're saying about all this and the module stuff especially - I commented in the other thread about one idea that combines functions and imports based on COBOL.

[Lartu]

Very interesting! I would use this in Dino, I have a few vectors that are n-dimensional using prefixes:

This was the original intention for using maps instead of arrays in LDPL. I though one would be able to imitate n-dimensional arrays using string indexes, but I guess overall it ended being not such a good idea.

SPLIT text BY " " IN myVector:prefix

I feel that something like this could be understood as "split text by " " in this particular index", but an index can't be more than one value! How come! It's not that I don't like it, but it would end up giving the SPLIT statement two different behaviors, one with vector and one with vector:index, when normally when vector:index is used in code it means a numeric or textual value.

Yes I like what you're saying about all this and the module stuff especially - I commented in the other thread about one idea that combines functions and imports based on COBOL.

I love what you said here. Way better than namespaces!

[xvxx]

Good ideas here, I'm going to close this unless you think it should stay open.

[Lartu]

Okay! :)