Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Win32: allow generating funcs.h and help.c without posix tools #446

Merged
merged 2 commits into from
Nov 6, 2023

Conversation

avih
Copy link
Contributor

@avih avih commented Oct 24, 2023

The first commit is for MSVC build, which, so far, didn't have a way to generate funcs.h and help.c.

Now it can generate them, by building a small C program which is then used instead of the posix tools:

nmake -f Makefile.wnm generated

Since we already have this C program, the 2nd commit adds support also for the mingw makefile, which allows building on windows even in environments without posix tools - like several mingw distributions, e.g. llvm-mingw or winlibs, by adding WINGEN=1 to the make invocation.

@avih
Copy link
Contributor Author

avih commented Oct 24, 2023

@Konstantin-Glukhov I recall you didn't have a way to build a git snapshot due to missing funcs.h and help.c. This PR allows building them using nmake -f Makefile.wnm generated.

Can you please try it out and report back if this solves the msvc makefile issue to build a git snapshot?

@avih
Copy link
Contributor Author

avih commented Oct 24, 2023

Pushed a minor no-op readability improvements at buildgen.c at the first commit.

Previously the mscv Makefile.wnm didn't have rules to generate funcs.h
and help.c, meaning that it could only be used to build release
tarballs (which already have these files), but not git snapshots.

Now we have a new independent rule "generated" (not part of "all"),
which generates them. Because typical msvc setups do't have the tools
which are used elsewhere to generate these files, this commit adds
buildgen.c which, when compiled, can be used instead.

It's not part of "all" because when cross compiling, e.g. an arm
binary on Windows 64, buildgen.exe should be native and not arm,
so this allows making "generated" with a native toolchain, and then
"all" using the arm toolchain.
@avih
Copy link
Contributor Author

avih commented Oct 24, 2023

... and removed a stray space...

@avih
Copy link
Contributor Author

avih commented Oct 25, 2023

For what it's worth, I'm OK with pushing it as-is now, and addressing later any feedback which comes up, though I don't expecty many issues, if at all.

@Konstantin-Glukhov
Copy link
Contributor

nmake -f .\Makefile.wnm

Microsoft (R) Program Maintenance Utility Version 14.37.32822.0
Copyright (C) Microsoft Corporation. All rights reserved.

    del defines.h
    copy defines.wn defines.h
    1 file(s) copied.

NMAKE : fatal error U1073: don't know how to make 'funcs.h'
Stop.

@avih
Copy link
Contributor Author

avih commented Oct 27, 2023

Now it can generate them, by building a small C program which is then used instead of the posix tools:

nmake -f Makefile.wnm generated

@Konstantin-Glukhov I recall you didn't have a way to build a git snapshot due to missing funcs.h and help.c. This PR allows building them using nmake -f Makefile.wnm generated.

@avih
Copy link
Contributor Author

avih commented Oct 27, 2023

@Konstantin-Glukhov note this is not yet merged. To try it out, you need to:

  1. Download the patch https://github.com/gwsw/less/pull/446.patch
  2. Apply the patch: at the source dir of less, do: git am path/to/446.patch.
  3. Whenever you need to [re]generate funcs.h and help.c: nmake -f Makefile.wnm generated

@avih
Copy link
Contributor Author

avih commented Oct 30, 2023

@Konstantin-Glukhov
Do you plan to try it out?

@Konstantin-Glukhov
Copy link
Contributor

Why don't you want to do commit?

@Konstantin-Glukhov
Copy link
Contributor

I tested the patch and it seems like it successfully generated help.c and funcs.h. Building the targets with the generated files was also successful.

@Konstantin-Glukhov
Copy link
Contributor

One comment, why does it need to be a separate target: "generated". Can't you just make it as default build?

@avih
Copy link
Contributor Author

avih commented Oct 30, 2023

why does it need to be a separate target: "generated"

The reason is explained both at the commit message and at the patch. I'm open to other suggestions which will still solve the described issue.

@avih
Copy link
Contributor Author

avih commented Nov 1, 2023

For what it's worth, to make it generate funcs.h and help.c by default (as part of all), two issues need to be addressed.

The first is mentioned at the commit message, and that's cross compiling, e.g. to arm. The problem is that buildgen.exe should be native, even when the compiler is for arm.

And so, if generated was a dependency of the obj files, then in a clean repo (e.g. right after clone), nmake -f Makefile.wnm would succeed in a native compile env, but fail in a cross compile env, because buildgen.exe would target arm, and so it can't be executed to generate funcs.h and help.c.

However, this can actually be solved, by splitting the generated section into two parts:

funcs.h help.c:
	buildgen.exe help < less.hlp > help.c
	type *.c 2>NUL | buildgen.exe funcs > funcs.h

buildgen.exe: buildgen.c
	$(CC) buildgen.c

So in a native compile env, this would suffice: nmake -f Makefile.wnm to both generate the files and then do the build.

And for cross compiling, we'd need to first use a native environment for buildgen: nmake -f Makefile.wnm buildgen.exe, and then with the arm cross environment run nmake -f Makefile.wnm.

And because buildgen.exe was already built natively, and it's newer than buildgen.c - it will NOT get built again with the arm compiler, but will still be used to generate possibly updated funcs.h and help.c.

The second problem is that we still want the build to be incremental. Currently it is incremental, and changing one c file and then nmake -f Makefile.wnm would indeed only compile this file and then link less.exe.

But if we add funcs.h as a dependency, and funcs.h is generated every time, then all the C files would get rebuilt (because all the objects depend on funcs.h).

The mingw makefile solves it by generating a new funcs.h, then comparing it with the old one, and then copy the new over the old only if they differ. So if they're the same, the date of funcs.h is not modified, and the C files are not rebuilt.

But this doesn't work with nmake. The part of generating a new and copying over only if they differ DOES work, but then all the C files are rebuilt even if funcs.h didn't actually change.

So I couldn't solve this problem.

For reference, that's my best attempt, which still rebuilds all C files whenever one C file changes (the rest or Makefile.wnm is unmodified, because it already depends on help.c and funcs,h):

help.c: buildgen.exe less.hlp
	buildgen.exe help < less.hlp > help.c

funcs.h: buildgen.exe *.c
	type *.c 2>NUL | buildgen.exe funcs > funcs.h.tmp
	comp /M funcs.h.tmp funcs.h || move funcs.h.tmp funcs.h

buildgen.exe: buildgen.c
	$(CC) buildgen.c

But overall, the instructions are fairly clear, and it's not hard to run nmake -f Makefile.wnm generated before running the build itself.

That's also not unlike on *nix where Makefile.aut should be used first to generate funcs.h and help.c (and the unicode files and configure etc, which we don't do on windows).

So I think this solution should be acceptable, and most importantly, it does cover the needs.

@gwsw thoughts? reasons for not commenting (or merging) so far?

We already have buildgen.c which can be used to generate funcs.h and
help.c without grep/sed/perl/sh/etc, which can be useful in mingw
setups on Windows which don't bring the whole posix arsenal with them.

The default is still the posix tools, but now adding WINGEN=1 will
use buildgen.c instead of the posix tools.

The compiler which is used with buildgen.c is the same compiler used
at the rest of Makefile.wng, so it should be native (not cross build).
@avih
Copy link
Contributor Author

avih commented Nov 1, 2023

Updated a comment at the mingw makefile about cross compiling without posix tools.

@gwsw gwsw merged commit f7d135e into gwsw:master Nov 6, 2023
@avih avih deleted the win32-build-gen branch November 6, 2023 21:10
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants