servestatic (SPIFFS) method will not send the whole file (tested with 30kB file)

[maichaell]

Hardware

Hardware: ESP-12E NodeMcu 1.0
Core Version: latest GIT

Description

Problems is that when using servestatic method esp8266 server does not send all content and browser stays at 'Connecting....'

I have uploaded files :

index.html
jquery-3.2.0.min.js.gz

into flash using ESP8266FS tool from Arduino IDE.

It seems index.html is sent okay but when ESP8266 server tries to send jquery-3.2.0.min.js.gz file it only sends part of it and not everything. Browser will stay 'connecting'

I this known error ? Can I enable some debugging to see better what is going on ?

Settings in IDE

Module: NodeMCU 1.0
Flash Size: 4MB/1MB
CPU Frequency: 80Mhz
Flash Mode: qio
Flash Frequency: 40Mhz
Upload Using: SERIAL
Reset Method: nodemcu

Sketch

#include <Arduino.h>

void setup(void){
  pinMode(led, OUTPUT);
  digitalWrite(led, 1);
  Serial.begin(115200);
  SPIFFS.begin();
  
  Serial.println();
  
    Dir dir = SPIFFS.openDir("/");
    while (dir.next()) {
      String fileName = dir.fileName();
      size_t fileSize = dir.fileSize();
      Serial.printf("FS File: %s, size: %s\n", fileName.c_str(), String(fileSize).c_str());
   
   Serial.println();
}
  
  WiFi.mode(WIFI_AP);

  WiFi.softAP(ssid);
  delay(200);

  IPAddress myIP = WiFi.softAPIP();

  Serial.println("AP IP address: ");
  Serial.println(myIP.toString()); 
        
  server.on("/post", HTTP_POST, handleRootPOST);
  
  server.on("/inline", [](){
    server.send(200, "text/plain", "this works as well");
  });  
  
  server.serveStatic("/index.html", SPIFFS, "/index.html");
  server.serveStatic("/jquery-3.2.0.min.js.gz", SPIFFS, "/jquery-3.2.0.min.js.gz");  
  server.serveStatic("/", SPIFFS, "/index.html");

  server.begin();
  Serial.println("HTTP server started");
}

void loop() {
  server.handleClient();
}

[maichaell]

Capture from WireShark using 'Follow TCP Stream'

[d-a-v]

Out of curiosity, could you try #3027 ?

[maichaell]

Yes! but can you help in merging. Seems I've forgotten something..

$ git fetch origin pull/3073/head
remote: Counting objects: 7, done.
remote: Compressing objects: 100% (7/7), done.
remote: Total 7 (delta 1), reused 0 (delta 0), pack-reused 0
Unpacking objects: 100% (7/7), done.
From https://github.com/esp8266/Arduino

• branch refs/pull/3073/head -> FETCH_HEAD

git checkout FETCH_HEAD/3073/head
error: pathspec 'FETCH_HEAD/3073/head' did not match any file(s) known to git.

[d-a-v]

I tried too that way and I failed.
The following works if you want to try:

git remote add tcptest https://github.com/d-a-v/Arduino.git
git pull tcptest tcpbufferized

when you are done:

git remote prune tcptest
git remote remove tcptest
git checkout origin

[maichaell]

Got it. I'll try it.

Do you know how to enable DEBUGV macros if you need I can enable them ? I did it a year ago but can't remember how to do it anymore :-)

[d-a-v]

I don't know the proper way, but adding #define DEBUG_ESP_CORE
on top of cores/esp8266/debug.h
does the trick.

[maichaell]

I just flashed the new image but unfortunately the situation is the same. Browser stays at 'Connecting..'. Perhaps my issue could be something else ?

I still don't see any debug messages which I find quite strange. I'll try to get them displayed.

[maichaell]

Finally got debug working. It seems you have to also call Serial.setDebugOutput(TRUE);

Same thing, it jammed but hopefully this debug will reveal something.

Here's log from Serial Monitor:

FS File: /index.html, size: 583
FS File: /jquery-3.2.0.min.js.gz, size: 30023

AP IP address: 
192.168.4.1
StaticRequestHandler: path=/index.html uri=/index.html isFile=1, cache_header=<null>
StaticRequestHandler: path=/jquery-3.2.0.min.js.gz uri=/jquery-3.2.0.min.js.gz isFile=1, cache_header=<null>
StaticRequestHandler: path=/index.html uri=/ isFile=1, cache_header=<null>
HTTP server started
add 1
aid 1
station: 24:77:03:47:99:34 join, AID = 1
WS:ac
:ref 1
WS:av
:ref 2
:ur 2
:ref 2
:ur 2
:rfb0
:rfb 0, 430
StaticRequestHandler::handle: request=/ _uri=/
StaticRequestHandler::handle: path=/index.html, isFile=1
:w 84
:w 256
:w 256
:w 71
SPIFFS_close: fd=1
:sent 84
:sent 583
:rcl
:abort
:ur 1
WS:dis
:del
WS:ac
:ref 1
WS:av
:ref 2
:ur 2
:ref 2
:ur 2
:rfb0
:rfb 0, 392
StaticRequestHandler::handle: request=/jquery-3.2.0.min.js.gz _uri=/jquery-3.2.0.min.js.gz
StaticRequestHandler::handle: path=/jquery-3.2.0.min.js.gz, isFile=1
:w 95
:w 256
:w 256
:w 256
:w 256
:w 256
:w 256
:w 256
:w 256
:w 256
:w 256
:w 256
:w 9
:sent 95
:w 95

[d-a-v]

The debug info reveals that esp stores your data in the outgoing tcp queue (:w) but are not actually sent to the client. Only some got out from esp (:sent).
I have no real clue of what's going on.
You could use wireshark to show the packet exchanges (not the stream) to understand why tcp protocol is not behaving.

[maichaell]

Attached wireshark log:

wireshark_log.zip

[maichaell]

Ok, now I tried connecting with my cell phone and it works !!

There must be something wrong with my laptop WLAN adapter I guess..

here's log with connecting from cell phone

add 2
aid 2
station: b4:e1:c4:3c:7c:08 join, AID = 2
WS:ac
:ref 1
WS:av
:ref 2
:ur 2
:ref 2
:ur 2
:rfb0
:rfb 0, 370
StaticRequestHandler::handle: request=/ _uri=/
StaticRequestHandler::handle: path=/index.html, isFile=1
:w 84
:w 256
:w 256
:w 71
SPIFFS_close: fd=1
:sent 84
:sent 583
:rcl
:abort
:ur 1
WS:dis
:del
WS:ac
:ref 1
WS:av
:ref 2
:ur 2
:ref 2
:ur 2
:rfb0
:rfb 0, 407
StaticRequestHandler::handle: request=/jquery-3.2.0.min.js.gz _uri=/jquery-3.2.0.min.js.gz
StaticRequestHandler::handle: path=/jquery-3.2.0.min.js.gz, isFile=1
:w 95
:w 256
:w 256
:w 256
:w 256
:w 256
:w 256
:w 256
:w 256
:w:sent 95
:w 256
:w 256
:w 104
 256
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 256
:w 256
:w 256
:w 256
:w 180
:sent 1460
:w 256
:w 46
SPIFFS_close: fd=1
:sent 1460
:sent 302
:ur 1
:close
WS:dis
:del

[d-a-v]

Glad it worked, but I would'nt suspect your laptop adapter.
I too had strange tcp behaviour which I try to understand.
Had you try with your cell phone without the patch above ?
It can be disabled without recompiling with
extern int lwip_bufferize; somewhere in the top of the sketch,
and lwip_bufferize = 0; in setup().

edit: I meant "without switching branch", not "without recompiling"

[maichaell]

Thanks. I tried and set lwip_bufferize to 0 and recompiled/flashed. With my cell phone that works also.

So it seems esp is having some problems with my laptop wlan adapter. If you have some ideas I'm happy to debug more.

[d-a-v]

I have something else to try, could you send me an email

[RichardPar]

@maichaell @d-a-v

Was this gremlin resolved? I have the same issue and trying to understand if its my page or the ESP device. (Serving jquery, Bootstrap and some other CSS/javascript files that are quite large - 200k and above)

I don't believe its a laptop wifi issue at this time as two seperate laptops have the same problem..

Richard

[d-a-v]

It was resolved with what became this pull request.