Prepare inclusion of netlist.lua into make generated target.

[couriersud]

This change affects the mame build system so I am asking for a review from somebody familiar with the genie build system.

The local netlist make generated target parses all "*.cpp" files in the devices, analog and macro directories and creates code files in the generated directory. Adding a new device thus boils down to putting it in one of these folders.
But the file still needs to be added to netlist.lua.

This pull request is a first step so that the make generated target can create the list of files needed by netlist.lua going forward.

[cuavas]

You can do that by calling out to a script and executing the result from Lua the way we use makedep.py to generate the the Lua necessary to make SOURCES= builds work. You don’t need to do it as part of the generated target in the top-level makefile. See here in scripts/genie.lua where it executes the output of a Python script to scan the specified source files and enable the correct devices.

However, I’m not sure this is a good idea. It causes various issues:

• When files are added or removed, developers need to regenerate project files manually, whereas the project files will be automatically regenerated if the list of source files is updated in a Lua file. After doing a git pull and receiving added/removed/renamed netlist files your next build will fail with a compilation error (deleted/renamed) or a link error (added), and you’ll have to touch makefile or REGENIE=1 and rebuild. This will be an annoyance for regular developers, and an additional support burden for dealing with people who build occasionally. Note that this is a big enough issue that CMake has a warning in the documentation about it on this page: “Note We do not recommend using GLOB to collect a list of source files from your source tree. If no CMakeLists.txt file changes when a source is added or removed then the generated build system cannot know when to ask CMake to regenerate.”
• If a file is inadvertently deleted, you won’t realise until later in the process. Rather than getting an error from the build tool when it’s unable to find the source to build the object, you get a link error at a later stage.
• If you do something like create a source file for testing, or rename/copy a source file out of the way (e.g. copy file.cpp to file_orig.cpp while working on file.cpp so you can easily copy/paste blocks from the original), the wile will get compiled and linked in.

Scanning for sources makes things less explicit, it introduces a source of spurious build failures after updating source, it opens developers up to more pitfalls, and it’s not the recommended way to add sources to projects in any build system, with CMake going so far as to explicitly recommend against doing it. Yeah, I know we do it for layout files, but even there I’m not really comfortable with it, but we can’t get proper dependencies for them anyway because GENie lacks an #include scanner.

(Unless you’re planning to scan for sources on every single build, generate a temporary Lua file, compare its contents to the existing one, copy it over if different, and make the project files depend on that. But that’s ugly to do with make lacking a convenient copy-if-different helper, and the directory scanning could slow builds down on Windows where filesystem overhead is significant.)

[cuavas]

Also, premake 4 and premake 5 support globbing for files natively similarly to what CMake does (with the same potential pitfalls) – see here for examples. Does GENie have that functionality?

[couriersud]

src/lib/netlist/build/makefile has it's own generated target. After adding a new source to the devices/analog/macro directories netlist developers need to

cd src/lib/netlist/build/makefile
make generated

to create all the additional code they previously needed to add manually. This approach is the result of feedback I received that adding devices require changes in too many places.

I fully understand the points you are making. This is why I made this a pull request - it didn't look convincing but I had no better solution. I can also create scripts/src/netlist.lua automatically with the other files with an explicit file list in it.
Would that be a better approach?

[cuavas]

Give me a bit of time to think about it some more. I understand that none of the options are completely ideal.