Server discovery on local network

[louisroyer]

Problem

I run a private minetest server on my local network for familial usage. When someone at home wants to connect to my server, it is mandatory to know it's address and add manually the server before connecting for the first time. I expected minetest clients to automatically discover servers on local network.

Solutions

Client could broadcast a discover request, and if server is configured to, it would respond with the server_name, server_description and server_address + port.
Then it would appears in the same list as public servers (with an icon saying it is a server from the local network).

[hecktest]

Yes, this is pretty standard and we should probably have it.

As a workaround, you can make launcher scripts for your family using the --address commandline param to avoid having to punch in the IP. Use a static IP for the server machine if you aren't already.

[rubenwardy]

I've wanted to do this for a long time, but never gotten around to it.

This implementation is very simple, the client sends a broadcast message (255.255.255.255) and the server responds with serverlist-like information (address, port, name, description)

[louisroyer]

As a workaround, you can make launcher scripts for your family using the --address commandline param to avoid having to punch in the IP. Use a static IP for the server machine if you aren't already.

Once the server is set as favorite, it is ok since I already use a static IP and the client will remember it. And for the desktop version I usually give them an mdns address (<hostname>.local), but sadly this is not supported in android.

This implementation is very simple, the client sends a broadcast message (255.255.255.255) and the server responds with serverlist-like information (address, port, name, description)

Yes and a multicast message (ff02::1) for ipv6.

[BruceMustache]

This implementation is very simple, the client sends a broadcast message (255.255.255.255) and the server responds with serverlist-like information (address, port, name, description)

@rubenwardy did you complete this feature? Will this be available in the next release?

[rubenwardy]

It won't, which is why this issue is still open. I have other priorities and I'm not interested in Minetest currently, anyway

Also, the protocol implementation is simple but actually integrating it into Minetest will probably be harder

[proller]

https://github.com/freeminer/freeminer/blob/master/src/network/fm_lan.cpp

[DustyBagel]

I would like to take a crack at this myself seeing as no one currently is working on it, but I am new to how minetest works so if someone can point me to a place to learn how the current network code works. Also, if someone could better explain to me how exactly we could implement this, that would be great. I don't know a lot about C++ or how minetest works, but I would like to try at least.

[DustyBagel]

So, do we just want to edit the server class and give it another asynchronous task of sending out the server information to a multicast address and then have the client listen on that address for the server information and update the server list accordingly? Wouldn't we also need a function in Lua we could call to do the listening sense all the Ui stuff is done in Lua now?

[appgurueu]

If you're new to C++ and Minetest dev, I'm afraid this might not be a "good first issue" for you to work on / get you started. It might be a better idea to practice C++, or to tackle some simpler issues first.

Anyways, as for your question: As has been said already, the client should send a broadcast message. The servers should then respond by sending their server info. The client needs to process these responses to integrate them into the server list. The UI would need to be update somehow, probably through a callback.

[proller]

just copy code from https://github.com/freeminer/freeminer/blob/master/src/network/fm_lan.cpp and all referring api/lua code around, it tested and works 8 years on all platforms

[rubenwardy]

That file says GPLv3 which is an incompatible license to LGPLv2.1+. So we'd need your permission to use it as LGPLv2.1+

[proller]

yes, I give permission to use all my freeminer code in minetest.
also it was in https://forum.minetest.net/viewtopic.php?p=283862#p283862

[DustyBagel]

This seems to do most of the heavy lifting. My only question would be how I would register a function/callback we can use for retrieving the server info so we can update the Ui. Do you already have a solution for that @proller? Also, how would I credit you correctly?

[proller]

credit: proller <[email protected]>

all needed changes:
https://github.com/freeminer/freeminer/blob/master/src/network/fm_lan.h
https://github.com/freeminer/freeminer/blob/master/src/network/fm_lan.cpp
https://github.com/freeminer/freeminer/blob/master/src/script/lua_api/l_mainmenu.h#L68-L69
https://github.com/freeminer/freeminer/blob/master/src/script/lua_api/l_mainmenu.cpp#L1061-L1094
https://github.com/freeminer/freeminer/blob/master/builtin/mainmenu/tab_online.lua#L25-L28
https://github.com/freeminer/freeminer/blob/master/builtin/mainmenu/tab_online.lua#L163
https://github.com/freeminer/freeminer/blob/master/builtin/mainmenu/tab_online.lua#L168
https://github.com/freeminer/freeminer/blob/master/builtin/mainmenu/serverlistmgr.lua#L116
https://github.com/freeminer/freeminer/blob/master/src/server.h#L764
https://github.com/freeminer/freeminer/blob/master/src/serverlist.h#L37-L39
https://github.com/freeminer/freeminer/blob/master/src/serverlist.cpp#L226-L238
https://github.com/freeminer/freeminer/blob/master/src/client/clientlauncher.cpp#L582
https://github.com/freeminer/freeminer/blob/master/src/network/address.h#L52-L54
https://github.com/freeminer/freeminer/blob/master/src/network/CMakeLists.txt#L2

(you can search by "lan" if i forgot something)

[proller]

Next thing to do: add multi address support for servers in list, join ipv4/ipv6 addreses of one servers by uuid in one record with something like addresess:["1.2.3.4", "fe80::2b8b:8de7:ab01:9c2c%3"]
this will solve duplicated lan servers for every network interface and allow v4/v6 master server announce

[Desour]

Is freeminer's lan advertisement protocol documented, beside source code?

If I saw it correctly, it just sends json strings over a fixed port (29998) via broadcasts. And the packets don't include anything that identifies them as minetest/freeminer packets.

Imo it would be preferable to use a standardized protocol for service discovery.
(Something from here https://en.wikipedia.org/wiki/Zero-configuration_networking. Idk which one to choose.)

[louisroyer]

Imo it would be preferable to use a standardized protocol for service discovery.

+1 : broadcasting some messages over UDP with a non well-known port may cause us unexpected issues at some point, and sound like reinventing the wheel.

If we are really going into that way, we should imo at least document a short protocol specification and ask IANA to be assigned an UDP port number. To be compliant with RFC6335, the name minetest-disc could be used for this protocol. Please, ensure a protocol version field is included in the payload, because only 1 port number will be assigned for all the versions of the protocol.

Some free-software games written in C++ already have some server discovery features using standard protocols (for example, 0ad seems to use UPnP.) Perhaps we could ask their developers for feedback to help us make an informed choice? (how hard to implement, what protocol they use and why, issues they may have with this protocol, cross-platform support…)

[DustyBagel]

Don't count on me coming out with a pr for this. I am running into too many complier errors then I know how to fix.

[DustyBagel]

@proller fm_lan.cpp can't be easily integrated into minetest because it is expecting a different version of the address class. The current address class can't handle the arguemnts that are being passed to it. I am getting these compiler errors:
Severity Code Description Project File Line Suppression State Details
Error C2440 '': cannot convert from 'sockaddr_in' to 'Address' minetest C:...\minetest\src\network\fm_lan.cpp line 103

Error C2440 '': cannot convert from 'sockaddr_in6' to 'Address' minetest C:..\minetest\src\network\fm_lan.cpp line 158

Error C2440 '': cannot convert from 'initializer list' to 'Address' minetest C:..\minetest\src\network\fm_lan.cpp line 195

Any ideas on how we can fix this?

[proller]

yep, forgotten:
https://github.com/freeminer/freeminer/blob/master/src/network/address.h#L52-L54
also possible to get diff between mt 5.8.0 and freeminer for all or specific files:
git diff 49ce5a2 address.h address.cpp

[DustyBagel]

@proller What is the purpose of the Serverlist::addMultiProto function? I have almost completed implementing everything, I got the lan discovery to work, I am just working out some bugs. Currently the lan_adv::run function uses a setting called "server_proto" and sense this setting doesn't exist in minetest, that causes an unhandled exception. I am wondering if I should add the setting or get rid of everything using it sense it doesn't seem necessary to integrate this into minetest.

[proller]

@DustyBagel remove all addMultiProto and server_proto

Its for freeminer because it can handle multiple protocols on different ports
maybe it can be useful in far future when somebody makes new good incompatible network protocol for minetest

[DustyBagel]

@proller Ok, where do I put this code then?
collected.insert_or_assign(key, p); fresh = true;
This needs to be done at some point because the get_lan_servers function depends on the collected variable.

[proller]

Maybe better to hardcode proto to "mt" for future extends
If not -

- } else if (p["proto"] == proto) {
+} else {

[DustyBagel]

Would hardcoding it in for now until possibly adding it down the line make minetest more compatible with freeminer? If so, then we should probably do it that way, I don't know if compatibility with forks of minetest is a priority though.

[kno10]

Instead of an own protocol, it would make sense to use standard multicast DNS for zero-configuration networking, e.g., using Bonjour even though this may require using different APIs on OSX, Linux, Windows (I am not an expert, it may well be possible to just use plain UDP and use the right message format, and the purpose of Avahi).
The idea is roughly the same - you can send a UDP broadcast to discover services, but the answers are DNS resource records.