Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[String] Fix float assignment using wrong type #8663

Closed
wants to merge 1 commit into from

Conversation

TD-er
Copy link
Contributor

@TD-er TD-er commented Aug 25, 2022

Using 3.0.2 assigning a float to a string resulted in undefined behaviour.

String str;
float myfloat[12][4];
myfloat[0][0] = 1.0f;
str = myfloat[0][0];

str does contain a single char.

Not sure if a 2D float array is needed to reproduce it, but that's how it was used in my project.
These were part of a larger struct, so maybe the assignment should be done in a separate function to make sure the compiler is not optimizing the code too much.

Worked fine in 2.7.4
Looks like it might have been introduced here: #8526 @mcspr

Change is mainly to enforce explicit behavior when calling the toString static functions by using const reference as their argument.

Also added bool operator ==(const __FlashStringHelper *s) const and its counter part, which saved roughly 3k in build size on my project ESPEasy.

Using 3.0.2 assigning a float to a string resulted in undefined behaviour.

```c++
String str;
float myfloat[12][4];
myfloat[0][0] = 1.0f;
str = myfloat[0][0];
```

`str` does contain a single char.

Worked fine in 2.7.4
@TD-er
Copy link
Contributor Author

TD-er commented Aug 25, 2022

Hmm @tonhuisman mentioned to me that the operator== for flash strings was removed in another commit, I started looking for it in the history.

It was indeed removed here: #8569
Does it still need to be removed?

@mcspr
Copy link
Collaborator

mcspr commented Aug 25, 2022

You sure about the source of the issue? What about a device / host test? Notice the CI suite and the tests/ dir related to strings (...and let me check the issue as well :)
FlashStringHelper was intentionally removed, iirc pointer overload is a source of trouble. And pretty sure you could resort to using const char as-is anyway.

@mcspr
Copy link
Collaborator

mcspr commented Aug 26, 2022

Using 3.0.2

...and are these tests done on git installation?

@TD-er
Copy link
Contributor Author

TD-er commented Aug 26, 2022

Using 3.0.2

...and are these tests done on git installation?

Used this in the PlatformIO config:

[core_3_0_2]
extends                   = esp82xx_3_0_x
platform                  = [email protected]
platform_packages = 
    platformio/framework-arduinoespressif8266 @ https://github.com/esp8266/Arduino.git
    mcspr/toolchain-xtensa @ ~5.100300.211127
build_flags               = ${esp82xx_3_0_x.build_flags}
                            -DPIO_FRAMEWORK_ARDUINO_ESPRESSIF_SDK3
lib_ignore                = ${esp82xx_defaults.lib_ignore}
                            IRremoteESP8266
                            HeatpumpIR
                            LittleFS(esp8266)
                            ServoESP32
                            TinyWireM

I did for sure a clean build.
@tonhuisman did test it on several builds and also on the build I made with these changes.

This is the section which was our use case:
https://github.com/letscontrolit/ESPEasy/blob/ab9fa55bc9e7d5b457e69a685b9d4b243e3ee341/src/_P021_Level.ino#L127-L136
Line 132, where string got assigned a float.

By explicitly creating a string and assigning it to the string object, this issue was fixed in ESPEasy, but of course then we don't have a clue where else it may cause issues.

N.B. the operator== doesn't have anything to do with this fix, it was just added as way to reduce build size, but I guess if it does cause issues, I will remove it and go over my code to replace all == and != compares with flash strings to the equals function.

@mcspr
Copy link
Collaborator

mcspr commented Aug 26, 2022

I did for sure a clean build.

With the toString() changes included in the latest commits? PIO does not keep latest branch HEAD up-to-date automatically. That PR fixed the integer coercion like here, where it was incorrectly assumed to be a char type.

This is the section which was our use case:
https://github.com/letscontrolit/ESPEasy/blob/ab9fa55bc9e7d5b457e69a685b9d4b243e3ee341/src/_P021_Level.ino#L127-L136
Line 132, where string got assigned a float.

By explicitly creating a string and assigning it to the string object, this issue was fixed in ESPEasy, but of course then we don't have a clue where else it may cause issues.

The idea is to have some small test code we could run without the whole ESPeasy becoming a test case

@TD-er
Copy link
Contributor Author

TD-er commented Aug 26, 2022

With the toString() changes included in the latest commits?

I did edit the WString.cpp/.h files in my local branch of this repo and then copied those 2 files to the tree in the .pio/packages folder.
So it was for sure using the same files as I have here in the PR.

I also removed all pio related caches before testing.

The idea is to have some small test code we could run without the whole ESPeasy becoming a test case

Will try to make a suitable test case later this evening.

@mcspr
Copy link
Collaborator

mcspr commented Aug 26, 2022

Why I am asking, I don't see any issues with overload itself (that this PR proposes to change)
Also note this will break some libs that use bit-field members and serialize them won't work via references.

PIO repo for me is in ~/.platformio/packages/framework-arduinoespressif8266@src-31d658a59f41540201fc3726a1394910. No need to manually copy anything, just 'checkout' / 'switch' into the master branch and try to build some small project referencing git in 'platform_packages'. e.g.

[env]
framework = arduino

[env:d1_mini]
platform = espressif8266
board = d1_mini
build_flags = -g
platform_packages =
    mcspr/toolchain-xtensa @ ~5.100300.211127
    platformio/framework-arduinoespressif8266 @ git+https://github.com/esp8266/Arduino.git
#include <Arduino.h>

void setup() {
    Serial.begin(115200);
}

void loop() {
    String a(1.0f);
    printf("String ctor -> \"%s\"\n", a.c_str());

    String b;
    float num = 1.0f;
    b = num;
    printf("String assignment -> \"%s\"\n", b.c_str());

    delay(1000);
}
String ctor -> "1.00"
String assignment -> "1.00"

@TD-er
Copy link
Contributor Author

TD-er commented Aug 27, 2022

OK, made a quick MCVE to show what's wrong.
Also included the PIO config so you can see what exact env is used.
See: https://github.com/TD-er/MCVE_ESPxx/tree/bug/ESP8266_Arduino_8663_Str_assignment

#define COL 12
#define ROW 4

String str;
float myfloat[COL][ROW];
int idx;


void setup() {
  idx = 0;
  Serial.begin(115200);
  for (int i = 0; i < COL; ++i) {
    for (int j = 0; j < ROW; ++j) {
      myfloat[i][j] = 1.0f * i * j + j;
    }
  }
}

void loop() {

  str = myfloat[idx / COL][idx % ROW];
  Serial.print(F("str: '"));
  Serial.print(str);
  Serial.println('\'');
  delay(1000);
  ++idx;
  if (idx >= (COL*ROW)) {
    idx = 0;
  }

}
PLATFORM: Espressif 8266 (4.0.1+sha.d2f3b91) > Espressif ESP8266 ESP-12E
HARDWARE: ESP8266 80MHz, 80KB RAM, 1019.98KB Flash
PACKAGES:
 - framework-arduinoespressif8266 @ 3.30002.0 (3.0.2)
 - tool-esptool @ 1.413.0 (4.13)
 - tool-esptoolpy @ 1.30000.201119 (3.0.0)
 - tool-mklittlefs @ 1.203.210628 (2.3)
 - tool-mkspiffs @ 1.200.0 (2.0)
 - toolchain-xtensa @ 2.100300.210717 (10.3.0)
LDF: Library Dependency Finder -> https://bit.ly/configure-pio-ldf
LDF Modes: Finder ~ chain, Compatibility ~ soft
Found 36 compatible libraries
Scanning dependencies...
No dependencies
Building in release mode
Warning! '-Wl,-T' option for specifying linker scripts is deprecated. Please use 'board_build.ldscript' option in your 'platformio.ini' file.
Compiling .pio\build\MCVE_ESP8266_4M1M\src\main.cpp.o
Linking .pio\build\MCVE_ESP8266_4M1M\firmware.elf
Retrieving maximum program size .pio\build\MCVE_ESP8266_4M1M\firmware.elf
Checking size .pio\build\MCVE_ESP8266_4M1M\firmware.elf
Advanced Memory Usage is available via "PlatformIO Home > Project Inspect"
RAM:   [===       ]  34.2% (used 27976 bytes from 81920 bytes)
Flash: [==        ]  24.5% (used 256249 bytes from 1044464 bytes)
Building .pio\build\MCVE_ESP8266_4M1M\firmware.bin
Creating BIN file ".pio\build\MCVE_ESP8266_4M1M\firmware.bin" using "C:\Users\gijsn\.platformio\packages\framework-arduinoespressif8266\bootloaders\eboot\eboot.elf" and ".pio\build\MCVE_ESP8266_4M1M\firmware.elf"
Configuring upload protocol...
AVAILABLE: espota, esptool
CURRENT: upload_protocol = esptool
Looking for upload port...
Auto-detected: COM5
Uploading .pio\build\MCVE_ESP8266_4M1M\firmware.bin
esptool.py v3.0
Serial port COM5
Connecting....
Chip is ESP8266EX
Features: WiFi
Crystal is 26MHz
MAC: 2c:3a:e8:39:14:07
Uploading stub...
Running stub...
Stub running...
Configuring flash size...
Compressed 260400 bytes to 191175...
Writing at 0x00000000... (8 %)
Writing at 0x00004000... (16 %)
Writing at 0x00008000... (25 %)
Writing at 0x0000c000... (33 %)
Writing at 0x00010000... (41 %)
Writing at 0x00014000... (50 %)
Writing at 0x00018000... (58 %)
Writing at 0x0001c000... (66 %)
Writing at 0x00020000... (75 %)
Writing at 0x00024000... (83 %)
Writing at 0x00028000... (91 %)
Writing at 0x0002c000... (100 %)
Wrote 260400 bytes (191175 compressed) at 0x00000000 in 17.0 seconds (effective 122.4 kbit/s)...
Hash of data verified.

Leaving...
Hard resetting via RTS pin...
=============================================================== [SUCCESS] Took 24.73 seconds ===============================================================
--- Terminal on COM5 | 115200 8-N-1
--- Available filters and text transformations: colorize, debug, default, direct, esp8266_exception_decoder, hexlify, log2file, nocontrol, printable, send_on_enter, time
--- More details at https://bit.ly/pio-monitor-filters
--- Quit: Ctrl+C | Menu: Ctrl+T | Help: Ctrl+T followed by Ctrl+H
str: ''
str: '␁'
str: '␂'
str: '␃'
str: ''
str: '␁'
str: '␂'
str: '␃'
str: ''
str: '␁'

@TD-er
Copy link
Contributor Author

TD-er commented Aug 27, 2022

OK, I did check the content of WString.h/.cpp and indeed these are not using the latest versions.
I did manually update the WString files with the current ones on GitHub and it works with your code.
But this leaves the question, how to get the latest code from this repo while using PlatformIO?
Apparently I don't now.

@mcspr
Copy link
Collaborator

mcspr commented Aug 27, 2022

The idea here is to use framework-arduinoespressif8266 @ git+https://... package spec so the original version gets 'substituted' with the a different one
https://github.com/platformio/platform-espressif8266/blob/d2f3b915a416c300e521cb76149a40c72acd7264/platform.json#L48-L53
No relation to the 'platform's git , it could use the usual versioned one

It should show up in the PACKAGES list when runing, too

> pio pkg list -e d1_mini
Resolving d1_mini dependencies...
Platform espressif8266 @ 4.0.1 (required: espressif8266)
├── framework-arduinoespressif8266 @ 3.1.0-dev+sha.313b3c0 (required: git+https://github.com/esp8266/Arduino.git) <<<<< this is a git version
...

But, it is up to the user to manually keep it up to date when no commit or version (tag) are specified in the repo URI

> pio pkg update -e ...

idk how GUI / IDE interacts with updates, and whether there are any more commands or command line flags that help out with the internal package structure

While I simply get by and edit package files and maintain development branches right there at the path I mentioned above, it is also possible to use 'local folders / symlinks' (https://docs.platformio.org/en/latest/core/userguide/pkg/cmd_install.html?highlight=symlink#local-folder)

platform_packages =
    framework-arduinoespressif8266 @ symlink:https://c:/Users/you/Documents/PlatformIO/esp8266-Arduino

Or, I wonder it would be best to refer to specific commit / tag / ref via github built-in repo download feature. No confusion in versioning then, everyone building ESPeasy gets the same version

https://github.com/esp8266/Arduino/archive/313b3c07ecccbe6fee24aa9fa447c4aed16ca499.zip
https://github.com/esp8266/Arduino/archive/refs/tags/whatever.zip
https://github.com/esp8266/Arduino/archive/refs/heads/master.zip

@TD-er
Copy link
Contributor Author

TD-er commented Aug 27, 2022

@mcspr
Thanks, I changed it to this one:
https://github.com/TD-er/MCVE_ESPxx/blob/800099a4fd2fe724d42f2c4417713ce208a42979/platformio.ini#L220-L225

Now it is working in the MCVE code without the needed changes of this PR.
So it can be closed as it was indeed already fixed.

N.B. my suggested fix may also cause issues for other types like volatile int as those differ from the types added, so const reference will not match.

@TD-er TD-er closed this Aug 27, 2022
@TD-er TD-er deleted the bugfix/wstring_float_assignment branch August 27, 2022 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants