Configure python environment and install esptool

[btashton]

This configures a the python user installation environment and then installs the esptool dependency. Should resolve the nightly build issue and enable us to more easily install python dependencies in the ci scripts.

[xiaoxiang781216]

How about change the path to:
PYTHONUSERBASE=$prebuilt/pylocal
like other tools

[btashton]

done.

[xiaoxiang781216]

can we check some file doesn't exist before install esptool like we check $prebuilt/xtensa-esp32-elf/bin/xtensa-esp32-elf-gcc?

[btashton]

pip performs this check and will skip downloading if it already exists so I think we are good.

[Ouss4]

@xiaoxiang781216 Will this suffice to fix 437 ?
The POSTBUILD script also copies the bootloader and the partition table from the IDF directory.

[xiaoxiang781216]

@xiaoxiang781216 Will this suffice to fix 437 ?
The POSTBUILD script also copies the bootloader and the partition table from the IDF directory.

Yes, I thinks so.

[btashton]

I am validating this docker image against the the testing system with the xtensa config enabled. I'll comment with a successful run.

[btashton]

@Ouss4 @xiaoxiang781216 This does not fix that issue. My CI run shows the IDF_PATH not being set so it cannot find the bootloader. It does move past the issue where the esptool.py does not exist. I am kind of thinking that we at least for the CI generate these two binaries ahead of time that way we do not have to pull in the whole IDF system.

====================================================================================
Configuration/Tool: esp32-core/nsh
------------------------------------------------------------------------------------
  Configuring...
  Building NuttX...
------------------------------------------------------------------------------------
xtensa-esp32-elf-gcc: warning: /github/workspace/nuttx/boards/xtensa/esp32/esp32-core/scripts/esp32.template: linker input file unused because linking not done
chip/esp32_allocateheap.c:70:2: warning: #warning REVISIT heap. Do what with non-heterogeneous memory? [-Wcpp]
 #warning REVISIT heap.  Do what with non-heterogeneous memory?
  ^~~~~~~
chip/esp32_clockconfig.c: In function 'esp32_clockconfig':
chip/esp32_clockconfig.c:80:4: warning: #warning WARNING: Clock configuration disabled [-Wcpp]
 #  warning WARNING: Clock configuration disabled
    ^~~~~~~
common/xtensa_dumpstate.c: In function 'xtensa_dumpstate':
common/xtensa_dumpstate.c:230:2: warning: #warning REVISIT interrupt stack [-Wcpp]
 #warning REVISIT interrupt stack
  ^~~~~~~
4096+0 records in
4096+0 records out
4194304 bytes (4.2 MB, 4.0 MiB) copied, 0.0110545 s, 379 MB/s
dd: failed to open '/hello_world/build/bootloader/bootloader.bin': No such file or directory
make: *** [tools/Makefile.unix:455: pass2] Error 1

[btashton]

I think I have the bootloader and partition table sorted out at least for the docker build, which I want to figure out before fixing up the ci shell script. But I think there might be something wrong with the esp32 build. It never generated the SCRIPTOUT target so linking failed:

====================================================================================
Configuration/Tool: esp32-core/nsh
------------------------------------------------------------------------------------
  Configuring...
  Building NuttX...
------------------------------------------------------------------------------------
chip/esp32_allocateheap.c:70:2: warning: #warning REVISIT heap. Do what with non-heterogeneous memory? [-Wcpp]
 #warning REVISIT heap.  Do what with non-heterogeneous memory?
  ^~~~~~~
chip/esp32_clockconfig.c: In function 'esp32_clockconfig':
chip/esp32_clockconfig.c:80:4: warning: #warning WARNING: Clock configuration disabled [-Wcpp]
 #  warning WARNING: Clock configuration disabled
    ^~~~~~~
common/xtensa_dumpstate.c: In function 'xtensa_dumpstate':
common/xtensa_dumpstate.c:230:2: warning: #warning REVISIT interrupt stack [-Wcpp]
 #warning REVISIT interrupt stack
  ^~~~~~~
xtensa-esp32-elf-ld: cannot open linker script file /github/workspace/nuttx/boards/xtensa/esp32/esp32-core/scripts/esp32_out.ld: No such file or directory
make[1]: *** [Makefile:144: nuttx] Error 1
make: *** [tools/Makefile.unix:449: pass2] Error 

[Ouss4]

Maybe a different solution is to wrap the whole post build script with a binary config similar to CXD56xx and not worry about it when we are just building.

[btashton]

I'm about 90% of the way now so I think I would rather just finish it. I suspect we have other binary blobs like wifi that we might want compiled in to other tests. One of my later goals is to get artifacts from builds that run on QEMU and then feed them into that. Right now I have a BLOBDIR variable that can be set to provide these to the build.

[btashton]

Ok I have a build working now. I'll need to update this a fair amount, but also submit a PR against the OS as well first.
https://github.com/apache/incubator-nuttx-testing/runs/526223529

[btashton]

Lets get the Github Actions version sorted out #20 and then I can look at how to integrate it into the cibuild.sh script.

[xiaoxiang781216]

Maybe a different solution is to wrap the whole post build script with a binary config similar to CXD56xx and not worry about it when we are just building.

@Ouss4 and @btashton, please look at this PR:
apache/nuttx#612

[Ouss4]

Thanks @xiaoxiang781216
@btashton That doesn't have to interfere with what you are doing. If everything is sorted out, we can still enable the config for the CI and nightly build.