Recommend WSL on Windows 10 [Discussion]

[NoahAndrews]

At this point, I consider the batch scripts @IBNobody and I worked on to mostly be a failure. They've proven to be unreliable, too dependent on the environment they're being run in, and I've seen far too many examples of people having frustrating issues with them that I haven't been able to help them with. They can also produce misleading and confusing error messages. I've been pointing people to use the WSL for a while now. Eventually, I think we should make a proper replacement for the batch scripts, possibly with an environment in msys2. For now, the WSL method in Windows 10 is far more reliable, and is easy to set up.

I also cleaned up some things in the WSL instructions themselves.

Jack, don't merge this right away. I would like to see what people (particularly @IBNobody and @fredizzimo) have to say about these changes first.

[skullydazed]

I'll second building on windows to be a problem.

We have Dockerfile and the Vagrantfile (and I recently published clueboard/qmk_firmware to dockerhub, for anyone that wants a one-command solution to setting up a build environment) and I wonder if pushing those on people who want to build on windows isn't the better way to go.

Longer term the non-technical user will be accommodated through web based firmware builders (I have one in progress, @ezuk has already built one for the ergodox-ez, and there's another one based on TMK I've seen.) They can already use qmk-firmware-flasher to flash a .hex file they acquire from someone else or one of those web interfaces. This will remove the pain of setting up a build environment for them.

That leaves only technical (power) users and programmers who will need to setup a build environment. A lot of them will already have either Vagrant or Docker installed, so spinning up a QMK build environment in a linux VM will be pretty painless.

This means that windows support is only for people who want to build on windows and can/do not want to setup a linux VM for some reason. I'm not suggesting we abandon those users, but it may make sense to restrict our support to only WSL and making it the non-default option. Sort of a, "we recommend a linux VM, but if you really want to use windows here's what you need..."

[IBNobody]

I agree that WSL is the answer for Windows 10, but Windows 7 is still a problem. (I still run Win7 at home, and it is still the dominant OS.)

My opinion is that I miss the simplicity of TMK's compilation. it didn't need GIT. It didn't need CMP. It just needed GCC and MAKE and sometimes DFU. When we started to add all these fancy dependencies to the code, we started to make the installation process a chore. Many of those who wanted to compile keyboard code find the layers that they have to go through intimidating.

I think @ezuk had the right idea in building a web-based compiler to solve that.

And for those who wanted to tinker in Win7, the scripts got us part of the way. If we could modify the makefile rules to omit GIT and CMP when running in Windows, we could complete the chain and make things easy for them. (Plus, there is another issue with Git. I discovered that Git for Windows and GitHub Desktop do not play nice together. Since the makefile needs Git, it leaves me in quite the pickle.)

Regarding Vagrantfiles(@skullydazed): The Vagrantfile only works once out of the box before the VirtualBox utilities and the Vagrant VM clash. I added instructions to the readme on how to correct it, but it was very off-putting. I had looked into the Docker solution, but I never got it working enough to evaluate how well it performed in a low-bandwidth environment. (I went with WSL instead.)

[skullydazed]

@IBNobody: If you want to play with docker, after installing it try "docker run clueboard/qmk_firmware". That will download approximately 800mb of docker images then drop you into a qmk_firmware checkout. From there you can "git pull" to get any commits since the image was built and start compiling. If you use Dockerfile directly from the QMK repo you'll end up pulling down more data, and will have to wait for the image to be built after everything downloads.

[fredizzimo]

It should be quite easy to remove the CMP and GIT dependencies on windows. But I won't have time to do it myself, as I would rather concentrate on finally getting the Infinity Ergodox LCDs and LEDs working properly.

For CMP, we could use the FC as an alternative.

Git is used for two different purposes,

1. To generate a version number, which I assume is meant to be helpful for reporting bugs. That could easily be disabled by default. It also slows down the compilation quite a bit
2. To check that you have all the submodules correctly checkout out when building the ChibiOS keyboards. We could easily turn this into a warning, when GIT is not installed.

However there's another problem with mingw, the date.exe command crashes when mingw is not properly installed. This can also probably be fixed somehow. The date command is also only used for version generation, so maybe that could be disabled as well.

Then regarding WSL.

First of all, it looks like full libusb support is very far away, as there's no real response here thread. They are considering adding USB storage device support though, but that doesn't help us.

However, the good thing is that Microsoft has added support for running normal executables from the bash. This should allow us to call a native executable for flashing from the makefile.

So I would recommend that we implement that first, before asking users to move over.

[IBNobody]

@fredizzimo

I am able to use WSL to program via the dfu-programmer package that the install_dependencies.sh file installs.

Here is the full list of packages that it gets...

  apt-get install -y \
    build-essential \
    gcc \
    unzip \
    wget \
    zip \
    gcc-avr \
    binutils-avr \
    avr-libc \
    dfu-programmer \
    dfu-util \
    gcc-arm-none-eabi \
    binutils-arm-none-eabi \
    libnewlib-arm-none-eabi \
    git \
    diffutils

[fredizzimo]

@IBNobody,

Dfu-programmer can be installed just fine, however it's really strange if programming actually works for you.

The output I get looks like this

dfu-programmer: can't init libusb.
Segmentation fault (core dumped)
Error: Bootloader not found. Trying again in 5s.
dfu-programmer: can't init libusb.
Segmentation fault (core dumped)
Error: Bootloader not found. Trying again in 5s.
^Cmake[1]: *** [dfu] Interrupt

Maybe it's actually launching the windows version for you, and therefore works?

[NoahAndrews]

It's important to note that as of now, only people running Windows Insider builds have the feature that lets you run Windows programs from WSL. It won't be coming to the rest of us until the Creators Update lands sometime next year. I still think it's time to stop recommending the scripts to new W10 users, even if that means they have to use the GUI-based flasher. Once the Creators Update comes out properly, we can make sure that we support command-line flashing from WSL.

@fredizzimo, you mentioned "asking users to move over". I don't think we should be doing that at all. If people have a Windows-based build environment that works for them, then great. No reason to switch. Maybe just add a warning saying that they should look into setting up a new build environment with one of the other methods if they run into too much trouble with the MinGW/scripts-based one.

[jackhumbert]

I'm gonna merge this, but feel free to continue the discussion!