Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Standardize the entry-point for Julia execution #50974

Merged
merged 6 commits into from
Sep 3, 2023
Merged

Standardize the entry-point for Julia execution #50974

merged 6 commits into from
Sep 3, 2023

Conversation

Keno
Copy link
Member

@Keno Keno commented Aug 18, 2023

This is a bit of a straw-man proposal (though I think mergable if people agree) to standardize the execution entrypoint for Julia scripts. I think there's at least four different ways that people might run a script:

  • As julia main.jl
  • As a PkgCompiler sysimage, then calling the main entry point
  • As a PkgCompiler "app", with the magic julia_main function
  • As a StaticCompiler product with an explicit entrypoint specified on the API.

The main problem I have with all of these variants is that they're all different and it's kind of a pain to move between them. Here I propose that we standardize on Main.main(ARGS) as the entrypoint for all scripts. Downstream from that proposal, this PR then makes the following changes:

  1. If a system image has an existing Main.main, that is the entry point for julia -Jsysimage.so.
  2. If not, and the sysimage has a REPL, we call REPL.main (we could handle this by defaulting Main.main to a weak import of REPL.main, but for the purpose of this PR, it's an explicit fallback. That said, I do want to emhpasize the direction of moving the REPL to be "just another app".
  3. If the REPL code is called and passed a script file, the REPL executes any newly defined Main.main after loading the script file. As a result, julia behaves the same as if we had generated a new system image after loading main.jl and then running julia with that system image.

The further downstream implication of this is that I'd like to get rid of the distinction between PkgCompiler apps and system images. An app is simply a system image with a Main.main function defined (note that currently PkgCompiler uses julia_main instead).

@KristofferC
Copy link
Sponsor Member

KristofferC commented Aug 19, 2023

An app is simply a system image with a Main.main function defined (note that currently PkgCompiler uses julia_main instead).

Not completely true. You can bundle multiple executables with an app and each executable is given a mapping to the Julia function it invokes when started.

But that could be changed to give arguments to the new main function or something.

@brenhinkeller
Copy link
Sponsor Contributor

Anything you'd want us to change in StaticCompiler? (not that we have a ton of flexibility but..)

@Seelengrab
Copy link
Contributor

Seelengrab commented Aug 19, 2023

In AVRCompiler.jl, I've established the convention of having a module-centric main function. That is, I expect to receive a module with a main function as the entry point for user code, after initialization of the microcontroller runtime. Due to the environment this runs on, this doesn't take any arguments, so standardizing on an optional ARGS would be nice. Otherwise this requires a forced "this code needs to be designed for microcontrollers" point, which I've managed to avoid so far (barring missing/impossible functionality).

@brenhinkeller
Copy link
Sponsor Contributor

Yeah, at the low level we're pretty much tied to argc::Int, argv::Ptr{Ptr{UInt8}} in StaticCompiler but it might be possible we could have some higher-level convenience representation which gets converted to that

@Seelengrab
Copy link
Contributor

Seelengrab commented Aug 19, 2023

Addendum to the above: The reasoning for requiring a module in AVRCompiler.jl and not directly an entry function is because that way I can get a reference to global variables that may need interning as well (doesn't help a lot, but feels nicer to be able to check from there instead of always chasing through pointers..). Additionally, because there is no compilation of additional code on microcontrollers, having a "This is the main module of the compiled binary" makes sense semantically.

@Keno
Copy link
Member Author

Keno commented Aug 19, 2023

Anything you'd want us to change in StaticCompiler? (not that we have a ton of flexibility but..)

Not immediately. There's longer term thoughts on unifying all the modes of app creation behind a new user experience. This is part of the thinking in that direction, but anything new would start out as a new package first (that may depend on PackageCompiler/StaticCompiler/etc).

Due to the environment this runs on, this doesn't take any arguments, so standardizing on an optional ARGS would be nice.

I think making ARGS optional is fine.

Yeah, at the low level we're pretty much tied to argc::Int, argv::Ptr{Ptr{UInt8}} in StaticCompiler but it might be possible we could have some higher-level convenience representation which gets converted to that

I think it'd be fine to wrap that into an array abstraction. I deliberately left ARGS untyped.

base/client.jl Outdated
elseif isassigned(REPL_MODULE_REF)
ret = REPL_MODULE_REF[].main(ARGS)
else
error("No entry point defined and REPL not loaded.")
Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems like an API break, since we expect Julia to work correctly to run scripts via exec_options particularly when the REPL is not loaded.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm fine with falling back to exec_options for the time being, though we may want to re-arrange later once we figure out what the compiler interface looks like.

src/jlapi.c Outdated
JL_TRY {
size_t last_age = ct->world_age;
ct->world_age = jl_get_world_counter();
jl_apply(&start_client, 1);
ret = jl_unbox_int32(jl_apply(&start_client, 1));
Copy link
Sponsor Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This should probably have a type-assert here first

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

_start is internal, so I though it was fine, but might as well be defensive about it.

@Keno
Copy link
Member Author

Keno commented Aug 23, 2023

Capturing some discussion from this morning:

  • @KristofferC didn't like the notion of a system image with an entry point, since the way that PkgCompiler works is that it builds one big system image and then potentially has multiple executables that load it and call different entry points. My counter was that nothing here prevents things from doing just that, this just provides a streamlined way to do the standard thing. From there you can then do the multi-app thing either with a main function that looks at argv[0] or with separate executables as you do now (even if the sysimage does have an entrypoint, you can just load it as a library). It also potentially suggests that we should make the sysimages executable by default if they have an entrypoint, so that loading them with julia is optional. (An in the fullness of time, with better static compilation, julia could be written in julia also).
  • @JeffBezanson and @vtjnash did not like the notion of making ARGS optional in the signature. Instead, we should pass an empty ARGS array (or some other empty array-like type should that interfere with static compilation for some reason). It was also pointed out that ARGS can happen even in some embedded use cases where the bootloader passes a command line.
  • There was some discussion of whether this should be __main__ or some other name that people are less likely to have already used in the script. @JeffBezanson and I strongly argued to just use a plain main. The existing __ function that we have __init__ is something that ideally users never have to use, whereas this is something that we expect people to use regularly. Of course, there is some possibility of unexpected behavior around the transition if some pre-existing main is called unexpectedly, but presumably that can be addressed by a sufficiently prominent note in the release notes.
  • @JeffBezanson asked a question about how to distribute apps as a package rather than a main.jl script. I suggested that apps distributed as packages should export a main function and can then be launched with julia -e 'using MyApp'. However, I think it should in general be discouraged that packages can be used as both a library and app application explicitly, and I would suggest that libraries that want to ship example or helper cli applications should provide an appropriate bin folder.

@Keno
Copy link
Member Author

Keno commented Aug 23, 2023

That said, overall, I didn't hear any strong objections to this, so I'm gonna go ahead and clean this up and get it ready for merging.

@Seelengrab
Copy link
Contributor

It was also pointed out that ARGS can happen even in some embedded use cases where the bootloader passes a command line.

While that's true in general, requiring it makes things harder because it requires array & String support in my runtime. Having ARGS be optional means I could conceivably migrate to the standardized approach with relatively little effort. I don't know whether a special type for ARGS could accomplish the same thing though - it depends on the specifics.

@Keno
Copy link
Member Author

Keno commented Aug 23, 2023

array & String support in my runtime

No it doesn't the custom empty array type can be trivial without runtime requirements.

@Seelengrab
Copy link
Contributor

Seelengrab commented Aug 23, 2023

If that's the case, then I'd be happy - I just know that dealing with arrays and strings in regular Julia code is a bit of a pain, due to their interactions with the runtime due to their dynamic lengths. The idea for making it optional is half inspired by C, which doesn't mandate the argc/argv structure either (see here).

A specific singleton type for "empty ARGS" should be workable, though I suspect that would be just the same as having no arguments passed at all.

@MasonProtter
Copy link
Contributor

MasonProtter commented Aug 23, 2023

One thing I'd like to bring up related to this topic is that we don't currently have an ergonomic way to write a script that takes advantage of the pkgimages stuff. If we're going to discuss a common entrypoint, we should at least try and keep our eyes open for designs that would make it easier to streamline this kinda clunky process for having good precompilation in scripts.

Currently what you'd need to do is to create a local package project, dev it, then write all your script functions into that package project, and then write precompile directives into the package, and then finally write a script which basically just does main(args...) = MyScriptPkg.main(args...).

Additionally, the binary artifact (pkgimage) associated with MyScriptPkg is hidden in .julia, which makes sense for packages, but maybe not for localized scripts.

@Seelengrab
Copy link
Contributor

Additionally, the binary artifact (pkgimage) associated with MyScriptPkg is hidden in .julia, which makes sense for packages, but maybe not for localized scripts.

It also doesn't make really sense for (standalone) applications that you'd like to distribute without a dedicated/existing Julia installation.

@Keno
Copy link
Member Author

Keno commented Aug 23, 2023

One thing I'd like to bring up related to this topic is that we don't currently have an ergonomic way to write a script that takes advantage of the pkgimages stuff. If we're going to discuss a common entrypoint, we should at least try and keep our eyes open for designs that would make it easier to streamline this kinda clunky process for having good precompilation in scripts.

I think that is mostly orthogonal to this proposal and more a question of more granular caching, which has been talked about but I don't think there are any concrete plans.

@Keno Keno added the kind:minor change Marginal behavior change acceptable for a minor release label Aug 28, 2023
This is a bit of a straw-man proposal (though I think mergable if
people agree) to standardize the execution entrypoint for Julia
scripts. I think there's at least four different ways that people
might run a script:

- As `julia main.jl`
- As a PkgCompiler sysimage, then calling the main entry point
- As a PkgCompiler "app", with the magic `julia_main` function
- As a StaticCompiler product with an explicit entrypoint specified
  on the API.

The main problem I have with all of these variants is that they're
all different and it's kind of a pain to move between them. Here
I propose that we standardize on `Main.main(ARGS)` as the entrypoint
for all scripts. Downstream from that proposal, this PR then makes
the following changes:

1. If a system image has an existing `Main.main`, that is the entry
   point for `julia -Jsysimage.so`.
2. If not, and the sysimage has a REPL, we call REPL.main (we could handle
   this by defaulting `Main.main` to a weak import of `REPL.main`, but
   for the purpose of this PR, it's an explicit fallback. That said, I do
   want to emhpasize the direction of moving the REPL to be "just another app".
3. If the REPL code is called and passed a script file, the REPL executes
   any newly defined Main.main after loading the script file. As a result,
   `julia` behaves the same as if we had generated a new system image after
   loading `main.jl` and then running julia with that system image.

The further downstream implication of this is that I'd like to get rid of
the distinction between PkgCompiler apps and system images. An app is simply
a system image with a `Main.main` function defined (note that currently
PkgCompiler uses `julia_main` instead).
@Keno Keno changed the title RFC: Standardize the entry-point for Julia execution Standardize the entry-point for Julia execution Aug 28, 2023
@Keno
Copy link
Member Author

Keno commented Aug 28, 2023

Rebased, dropped RFC, addressed review comments, added docs, news, a couple simple tests. I think this is essentially ready.

NEWS.md Outdated
Comment on lines 19 to 23
* The entry point for Julia has been standardized to `Main.main(ARGS)`. When julia is invoked to run a script or expression
(i.e. using `julia script.jl` or `julia -e expr`), julia will subsequently run the `Main.main` function automatically if
such a function has been defined. This is identended to unify script and compilation workflows, where code loading may happen
in the compiler and execution of `Main.main` may happen in the resulting executable. For interactive use, there is no semantic
difference between defining a main function and executing the code directly at the end of the script. ([50974])
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* The entry point for Julia has been standardized to `Main.main(ARGS)`. When julia is invoked to run a script or expression
(i.e. using `julia script.jl` or `julia -e expr`), julia will subsequently run the `Main.main` function automatically if
such a function has been defined. This is identended to unify script and compilation workflows, where code loading may happen
in the compiler and execution of `Main.main` may happen in the resulting executable. For interactive use, there is no semantic
difference between defining a main function and executing the code directly at the end of the script. ([50974])
* The entry point for Julia has been standardized to `Main.main(ARGS)`. When Julia is invoked to run a script or expression
(i.e. using `julia script.jl` or `julia -e expr`), Julia will subsequently run the `Main.main` function automatically if
such a function has been defined. This is intended to unify script and compilation workflows, where code loading may happen
in the compiler and execution of `Main.main` may happen in the resulting executable. For interactive use, there is no semantic
difference between defining a `main` function and executing the code directly at the end of the script. ([50974])

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is talking about the behavior of the binary, not the semantics of the language, so lower case is the appropriate style.

@Seelengrab
Copy link
Contributor

Seelengrab commented Aug 29, 2023

Can/should this catch when two packages define & export a main, and warn accordingly? This will already warn due to the duplication/shadowing, but an explicit warning & what to do in the special case of main would be nice.

@Keno
Copy link
Member Author

Keno commented Aug 29, 2023

Can/should this catch when two packages define & export a main, and warn accordingly? This will already warn due to the duplication/shadowing, but an explicit warning & what to do in the special case of main would be nice.

I think that can be a tweak to the existing error message, but I also think that can be a separate PR.

@adienes
Copy link
Contributor

adienes commented Aug 29, 2023

just to make sure I understand --- when loading the symbol via using, then main still only executes after my script does right?

that is

using Hello

main(ARGS) println("goodbye") end

prints only "goodbye", right?

@Keno
Copy link
Member Author

Keno commented Aug 29, 2023

prints only "goodbye", right?

correct

@JeffBezanson JeffBezanson added the status:triage This should be discussed on a triage call label Aug 29, 2023
@davidanthoff
Copy link
Contributor

Add some sort of ugly hack to the to determine if Main.main has ever been called and if so don't run it again - just give a warning

My best guess is that we'll want some kind of special support for this stuff in the VS Code extension at some point (although I haven't really thought through what that would actually be) and we normally have to figure out things statically, rather than at runtime. So, I'm a bit nervous about a hack that relies on whether something has been called or not at runtime, as that might prove to be a criterion we don't have access to in the extension. Hacky things in Julia tend to cause a lot of grief downstream in tooling, so my vote would be for something simple first and foremost.

@vtjnash
Copy link
Sponsor Member

vtjnash commented Sep 6, 2023

What we were proposing is a global called Base.run_main_after::Bool = true. This can be changed and inspected by user code, but will be assigned (to false) automatically by any function named Main.main(), when run. This will be inspectable by tools such as code_typed(Main.main), as it will indeed be an explicit part of the IR.

@davidanthoff
Copy link
Contributor

Does that mean if someone has a script script.jl

function main()
   # Do something evil here
end

function my_main()
  # Do something good here
end

my_main()

that main would actually run if a user does julia script.jl?

@odow
Copy link
Contributor

odow commented Sep 6, 2023

that main would actually run if a user does julia script.jl?

Yes. But I think actually both functions would run, my_main() first and then main().

Magically running user-code is the wrong thing to do in a 1.X release, and it should not even be up for debate, opt-outs, or hacky work-arounds.

I think the two options need to be:

  1. __main__ (or extend the current __init__ to also cover scripts)
  2. Some sort of explicit opt-in, via Project.toml or @entrypoint

@mkitti
Copy link
Contributor

mkitti commented Sep 6, 2023

Could the opt-in be as simple as declaring compat julia = 1.11?

@Keno
Copy link
Member Author

Keno commented Sep 6, 2023

Another option that was brought up: Change the name of the driver executable. The compiler driver was gonna be juliac but it doesn't have to be. We could have jl and jlc as the drivers for example and add proper cli versioning support at the same time (keeping julia as a deprecated driver with frozen cli).

@kdheepak
Copy link
Contributor

kdheepak commented Sep 6, 2023

I compiled this commit from source. I can confirm what @odow mentioned, it runs mymain() first and essentially runs exit(main(ARGS)) at the very end.

I know this thread is about what to do instead, but I wanted to say that I find it surprising that it runs main(ARGS) if you execute julia -e "using HelloWorld". If I hadn't read the documentation, this would have caught me by surprise.

And given that it does run main(ARGS) for julia -e "using HelloWorld", I suspect users will also find it surprising that it doesn't run main(ARGS) if you execute julia -e "import HelloWorld".

╭─ ~/g/j/HelloWorld - 8e14322 
╰ cat src/HelloWorld.jl
───────┬───────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: src/HelloWorld.jl
───────┼───────────────────────────────────────────────────────────────────────────────────────────────────
   1module HelloWorld
   23export main
   4main(ARGS) = println("Hello World!")
   56end # module HelloWorld
───────┴───────────────────────────────────────────────────────────────────────────────────────────────────

╭─ ~/g/j/HelloWorld - 8e14322../julia --startup-file=no --project -e "using HelloWorld"
Hello World!

╭─ ~/g/j/HelloWorld - 8e14322../julia --startup-file=no --project -e "import HelloWorld"

It also only runs main(ARGS) only when the symbol is exported and when running it with using HelloWorld.

I get that this works by checking if isdefined(Main, :main) && !is_interactive but it feels like there's a number of ways for beginners to get tripped up on this. imho, this violates the principle of least surprise.

Also, doesn't this mean you can't have a script that loads multiple packages with main(ARGS) using using Package without triggering warnings? Or am I missing something here?

This is what happens when I test this:

╭─ ~/g/j/HelloWorld - 8e14322 
╰ cat test.jl
───────┬───────────────────────────────────────────────────────────────────────────────────────────────────
       │ File: test.jl
───────┼───────────────────────────────────────────────────────────────────────────────────────────────────
   1module testing
   23export main
   4main(ARGS) = println("testing")
   5end
   67using .testing
   89using HelloWorld
───────┴───────────────────────────────────────────────────────────────────────────────────────────────────

╭─ ~/g/j/HelloWorld - 8e14322../julia --startup-file=no --project test.jl
WARNING: both HelloWorld and testing export "main"; uses of it in module Main must be qualified

Thinking about this more, this does feel like it really would benefit from a standardized entrypoint treatment via Project.toml.

For beginners writing a script, it doesn't get simpler than it is already, i.e.:

println("hello world")

For anyone generating a package using Pkg.generate(), if a entrypoint is created in Project.toml, it will be as easy as it is in this PR to run code defined in an entrypoint in the package. And this can probably be done without having to export functions and can be made to work with julia -e "import HelloWorld" and julia -e "using HelloWorld". At that point the function name (main vs __main__) wouldn't really matter.

@Seelengrab
Copy link
Contributor

Seelengrab commented Sep 6, 2023

I'm starting to warm up to the @entrypoint/Project.toml idea. One additional advantage (that may have been mentioned and that I just missed) is that this would likely be easier for PkgCompiler to hook up multiple entrypoints.

From the POV of static compilation, using __main__ exclusively feels REALLY odd. As far as I know, pretty much only python uses that as its "main entrypoint", and even then only through convention. For static compilation though, this marking of an entrypoint really needs to be standardized, and not a convention - if we were consistent with our existing convention (of checking !isinteractive()), we wouldn't already be in this discussion in the first place :)

@davidanthoff
Copy link
Contributor

Another idea could be a different file extension for scripts that have main as their entry point (jlx or something), while traditional jl files keep not having an entry point.

At some level I find the idea of an entry-point function almost a bit at odds with the current idea of what a script in Julia is. Isn't the entry point just the global code in a Julia file, essentially? The whole concept that one might have a script file with lots of global code (that isn't type/function/global var definitions) that runs first, and then another function with a specific name that runs in a second phase seems a bit weird to me in general... Maybe something like a jlx file could be an option where you generally can only have type/function definitions, and then you must have a main in there to be able to run that file.

@MasonProtter
Copy link
Contributor

I think having to specify the entrypoint in the Project.toml sounds like way too much friction and jumping around. You then cant just write a script, you have to create a project for every script which is pretty crappy IMO.

@KristofferC
Copy link
Sponsor Member

KristofferC commented Sep 6, 2023

Yes, I would say Project.toml or any other file specification of an entry point is a non-starter for discussion. It would make it way too annoying to use.

One additional advantage (that may have been mentioned and that I just missed) is that this would likely be easier for PkgCompiler to hook up multiple entrypoints.

PackageCompiler already has support for this, it wouldn't really make things easier.

@Seelengrab
Copy link
Contributor

I think having to specify the entrypoint in the Project.toml sounds like way too much friction and jumping around. You then cant just write a script, you have to create a project for every script which is pretty crappy IMO.

Do we really need/want script style code to be shared as a fully compiled executable though? I think it's ok to seperate the two use cases - script style is just "run julia script.jl", whereas compilation to a binary is "define an entry point & manage your dependencies". It's not like arbitrary scripts with tons of global variables are a good candidate for static compilation, which I think the majority of cases of scripts are going to be.

IMO scripts should be kept small & self-contained, and once your script grows to a size that sharing it as a naively-runnable thing becomes a burden, that's exactly the point where you'd want to have dependency management, versioning & a well-defined entry point anyway. Placing that in Project.toml just makes sense to me 🤷

@kdheepak
Copy link
Contributor

kdheepak commented Sep 6, 2023

Another advantage of placing it in a Project.toml is that we can automate installation of scripts during Pkg.add("Package").

For example, python has entrypoints that can be defined in setup.py and pyproject.toml files, e.g.

[project.scripts]
my-cli = "my_package.my_module:main_cli"

If someone pip install my_package, they'll get a my-cli in their PATH that effectively runs the following code:

import sys
from my_package.my_module import main_cli
sys.exit(main_cli())

Rust has something similar where you can define a bin in Cargo.toml:

[[bin]]
name = "my-cli"
path = "src/main.rs"

and this builds a target called my-cli.

In both these cases it is a few lines of toml code and little to no friction in my opinion.

I agree with @Seelengrab; from my experience a lot of Julia users tend to work in Pluto or jupyter notebooks or use just a single script.jl file in VSCode using Ctrl + Enter or Julia-VSCode's Run command. I don't expect most of them will even opt in to entrypoints, main etc. A lot of their code is in the global scope and is exploratory in nature. When it starts becoming a script that they run semi regularly or when they intend to distribute it with their peers, they are typically encouraged to make it a package anyway.

If this kind of "entrypoint" feature were part of a Project.toml, anyone using Pkg.generate could either get entrypoints out of the box or possibly opt-in with one line. This could open up some neat use cases like:

$ julia -e 'using Pkg.add("Genie")'
$ genie server

imo, one of the reasons Python is so popular for writing command line tools is that it makes it extremely easy to be able to pip install package and get a developer defined command line interface for interacting with a domain. Some fairly popular examples:

In the example of youtube-dl I bet a number of the users don't even know they are using Python. This kind of feature (along with some juliaup shenanigans) will make Julia a viable (or even preferred) alternative to Python for writing command line applications.

@caleb-allen
Copy link

We could have jl and jlc as the drivers for example and add proper cli versioning support at the same time (keeping julia as a deprecated driver with frozen cli).

I really like this. As a user this would certainly give me the impression that the execution of julia code may be different depending on how I invoke it. Separating the drivers implies to users (new and old) that executing a Julia file with jlc is, in some substantive way different from executing it with jl, even if the specifics are unclear; if I had just started using Julia and was running some old Julia code successfully with jl but got unexpected behavior with jlc I wouldn't blame julia per se, I would think that my code doesn't conform to whatever it is that jlc expects.

I think introducing entry point changes with jl and jlc drivers (perhaps with warnings about main where relevant) is a great solution.

@jonas-schulze
Copy link
Contributor

A new driver sounds reasonable, whether it's named jl or whatnot sounds like bike-shedding to me. I just don't like that the julia driver would be considered deprecated. The opt-in could be done with a shebang #!.

Keno added a commit that referenced this pull request Sep 21, 2023
Following the discussion in #50974, it became clear that there is significant
objection to changes to the behavior of the julia CLI driver. Some
commenters found changes to the behavior acceptable if they were unlikely
to impact existing code (e.g. in the case of #50974 using `__main__`
instead of `main`), while other were worried about the reputational
risks of even changing behavior in the corner case. In subsequent
discussion it became relatively clear that the only way forward
that did not raise significant objection was to introduce a new CLI
driver for the new behavior. This may seem a bit drastic just for the
change in #50974, but I actually think there's a number of other
quality-of-life improvements that this would enable us to do over time,
for example:
 - Autoloading/caching of all packages in the manifest
 - auto-selection of julia versions (integrate juliaup?)

In addition, it doesn't seem so bad to have some CLI design flexibility
to make sure that the `juliac` driver is aligned with what we need.

This PR is the minimal infrastructure to add the new drivers.
In particular, it does the following:

1. Adds two new cli drivers, `juliax` and `juliac`. At the moment,
   `juliac` is a placeholder and just errors, while `juliax` behaves
   the same as `julia` (except to error on the deprecated `--math-mode=fast`,
   just as an example of a behavior difference).

2. Documents that the behavior of `julia` (but not `juliax` or `juliac`)
   is pat of the julia public API.

3. Switches the cli mode based on the argv[0] of the binary. I.e. all three
   binaries are identical, except for their name, the same way that, e.g.
   `clang` and `clang++` are the same binary just with different names.
   On Unix systems, these are symlinks. On windows, separate copies of
   the same (small) binary. There is a fallback cli option `--cli-mode`
   that can be used in case the argv[0] detection is not available (e.g.
   for some fringe embedded use cases).

4. There is currently no separate `-debug` version of the new drivres.
   My intention is to make this dependent on the ordinary debug flags,
   rather than having a separate driver.

Once this is merged, I intend to resubmit #50974 (chaning `juliax` only),
and then finish and hook up `juliac` shortly thereafter.
@Keno
Copy link
Member Author

Keno commented Sep 21, 2023

#51417 for the new cli drivers, which emerged as the preferred option in subsequent discussion.

@PallHaraldsson
Copy link
Contributor

PallHaraldsson commented Sep 22, 2023

Was this breaking? I see first a "minor change" label, then label changed, and it's a long discussion, I think things changed here (why the new juliax PR), so it's best to change the label back if it wasn't actually breaking. In case people are searching through, it seems odd to see breaking (merged) PR, not on the 2.0 milestone. [Which has been renamed recently to "potential 2.0", I'm not sure, is it now more (or less) potential?]

@lgoettgens
Copy link
Contributor

This was first declared a minor change and then merged. Afterwards, it was decided that this change is breaking; thus it was reverted in #51196.

Keno added a commit that referenced this pull request Sep 23, 2023
As they say, if at first you don't succeed, try again, then
try again, add an extra layer of indirection and take a little
bit of spice from every other idea and you've got yourself a wedding
cake. Or something like that, I don't know - at times it felt like
this cake was getting a bit burnt.

Where was I?

Ah yes.

This is the third edition of the main saga (#50974, #51417). In
this version, the spelling that we'd expect for the main use case is:
```
function (@main)(ARGS)
    println("Hello World")
end
```

This syntax was originally proposed by `@vtjnash`. However, the semantics
here are slightly different. `@main` simply expands to `main`, so the above
is equivalent to:
```
function main(ARGS)
    println("Hello World")
end
@main
```

So `@main` is simply a marker that the `main` binding has special behavior.
This way, all the niceceties of import/export, etc. can still be used
as in the original `Main.main` proposal, but there is an explicit
opt-in and feature detect macro to avoid executing this when people
do not expect.

Additionally, there is a smooth upgrade path if we decide to automatically
enable `Main.main` in Julia 2.0.
Keno added a commit that referenced this pull request Sep 23, 2023
As they say, if at first you don't succeed, try again, then
try again, add an extra layer of indirection and take a little
bit of spice from every other idea and you've got yourself a wedding
cake. Or something like that, I don't know - at times it felt like
this cake was getting a bit burnt.

Where was I?

Ah yes.

This is the third edition of the main saga (#50974, #51417). In
this version, the spelling that we'd expect for the main use case is:
```
function (@main)(ARGS)
    println("Hello World")
end
```

This syntax was originally proposed by `@vtjnash`. However, the semantics
here are slightly different. `@main` simply expands to `main`, so the above
is equivalent to:
```
function main(ARGS)
    println("Hello World")
end
@main
```

So `@main` is simply a marker that the `main` binding has special behavior.
This way, all the niceceties of import/export, etc. can still be used
as in the original `Main.main` proposal, but there is an explicit
opt-in and feature detect macro to avoid executing this when people
do not expect.

Additionally, there is a smooth upgrade path if we decide to automatically
enable `Main.main` in Julia 2.0.
Keno added a commit that referenced this pull request Oct 8, 2023
As they say, if at first you don't succeed, try again, then
try again, add an extra layer of indirection and take a little
bit of spice from every other idea and you've got yourself a wedding
cake. Or something like that, I don't know - at times it felt like
this cake was getting a bit burnt.

Where was I?

Ah yes.

This is the third edition of the main saga (#50974, #51417). In
this version, the spelling that we'd expect for the main use case is:
```
function (@main)(ARGS)
    println("Hello World")
end
```

This syntax was originally proposed by `@vtjnash`. However, the semantics
here are slightly different. `@main` simply expands to `main`, so the above
is equivalent to:
```
function main(ARGS)
    println("Hello World")
end
@main
```

So `@main` is simply a marker that the `main` binding has special behavior.
This way, all the niceceties of import/export, etc. can still be used
as in the original `Main.main` proposal, but there is an explicit
opt-in and feature detect macro to avoid executing this when people
do not expect.

Additionally, there is a smooth upgrade path if we decide to automatically
enable `Main.main` in Julia 2.0.
Keno added a commit that referenced this pull request Oct 8, 2023
As they say, if at first you don't succeed, try again, then try again,
add an extra layer of indirection and take a little bit of spice from
every other idea and you've got yourself a wedding cake. Or something
like that, I don't know - at times it felt like this cake was getting a
bit burnt.

Where was I?

Ah yes.

This is the third edition of the main saga (#50974, #51417). In this
version, the spelling that we'd expect for the main use case is:
```
function (@main)(ARGS)
    println("Hello World")
end
```

This syntax was originally proposed by `@vtjnash`. However, the
semantics here are slightly different. `@main` simply expands to `main`,
so the above is equivalent to:
```
function main(ARGS)
    println("Hello World")
end
@main
```

So `@main` is simply a marker that the `main` binding has special
behavior. This way, all the niceceties of import/export, etc. can still
be used as in the original `Main.main` proposal, but there is an
explicit opt-in and feature detect macro to avoid executing this when
people do not expect.

Additionally, there is a smooth upgrade path if we decide to
automatically enable `Main.main` in Julia 2.0.
@ctarn
Copy link
Contributor

ctarn commented Apr 25, 2024

I would like to suggest a change to the doc to recommend users define main as main(args) instead of main(ARGS) to clarify the differences between the global var Core.ARGS and the function parameter. Defining the main(ARGS) is confusing to me if the ARGS is just a normal parameter.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
kind:breaking This change will break code
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet