-
Notifications
You must be signed in to change notification settings - Fork 2k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[no sq] Generic IPC mechanism between Lua envs #14659
base: master
Are you sure you want to change the base?
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
only looked at the documentation
I think channels, as done in love2d would be preferable to this. |
Channels are definitely more flexible, but there are two reasons that made me choose this approach:
|
nice work 👍 do i get this right: this is basically a persistent and global key/value store and i have to make sure the items don't pile up and fill my memory (as a modder)? |
Correct. |
What about the async environment? For my usecase I want a long-running task in its own thread. This is so I can run user-provided code in my (yet to be made, though i have implemented most of the VM for my fictional CPU architecture in Lua already) computer mod without blocking the main thread. The main thread and the code execution thread would communicate with each other by sending messages, where the main thread would send events to respond to and the VM in the execution thread would send actions it wants to perform in the world. Without an IPC mechanism i would have to serialize, send the entire VM state back and forth and reinitalize it on every event or action, which might be far too much overhead for this to be practical.
If the data is static, no IPC mechanism is needed. Something similar to the way its done in async env could be done for mapgen env, allowing modders to pass initial values. If more communication is needed, pass a channel to the env. Peekable channels could be used to store values. |
There is a fixed maximum amount of async workers, so if you block one of them you risk stalling all other jobs.
There is in fact no good way to pass static data into the async env. You can pass data together with every job, but then you pay the transfer cost every time. Regardless in #14451 OP requested the ability to change this data at runtime. So it's not really static. |
I think this limit should be removed. It's not even mentioned in the docs. Blocking only one of them also shouldn't stall anything as the work stealing algorithm will distribute the jobs among remaining workers. |
If the maximum of worker threads is 1, then blocking a single one will block everything.
|
:/ |
I changed my mind. People will find uses for it when it exists. |
891ecbe
to
7be58d9
Compare
It might also be useful to have an async polling function with a callback. With this one could do the same as checking a value in each server-step, but with less latency: core.ipc_set("mymod:please_write_it")
-- wait for value
local function wait_for_it()
local val = core.ipc_get("mymod:val")
if val == nil then
- core.after(0, wait_for_it) -- not called until next step
+ core.poll_async("mymod:val", nil, wait_for_it) -- called as soon as possible. (nil given for comparison, similar to futex API)
else
print(val)
end
end
wait_for_it() (Mods can also use it for synchronous polling by yielding a coroutine.) |
This would not share any code with the blocking implementation and have no advantages over manually polling with |
b2b25e7
to
842e042
Compare
Idk if correct or wrong, but a friend of mine suggested this solution is NIH and you could simply use sqlite instead because it is apparently thread-safe. |
To do
This PR is ready.
TODO:
How to test