HttpClient crashes after HTTP request

[KenthJ]

Board: ESP-WROOM-32
IDE name: Arduino IDE version 1.0.4
Flash Frequency: 80Mhz
PSRAM enabled: No
Upload Speed: 921600
Computer OS: Windows 10

Hi

Please bear with me, I'm completely new in this ESP32 world.

I'm trying to make a very simple HTTP request from my ESP32 towards a webserver which is running on a Raspberry Pi running Win 10 IOT, I have been banging my head for several hours and days trying to solve my problem below.

In most cases the code succeeds getting a response as expected, but when I call http.end() it crashes, sometimes with Guru Meditation Error: Core 1 panic'ed (InstrFetchProhibited), other times with (LoadProhibited).

If I instead use any other URLs instead of my own webserver, everything works fine.
When I test the URL of my Raspberry PI webserver with a normal internet browser on my PC, or using Fiddler everything works fine too.

The strange thing is that it only crashes when it sends a http req to my homemade webserver, it works fine with all other webservers.

I double checked my http header from the server, and also added "Content-Type: text/html" "Date:....." and "Server:..." to the header.

Further more I checked the header and html response using a program called Fiddler and using FireFoxs Inspector tool, and I really can't see any problem with it, but there must be something in the response from my webserver causing the crash

#include <WiFi.h>
#include <HTTPClient.h>
 
const char* ssid = "mySSID";
const char* password =  "myPassword";

void setup(void)
{
  Serial.begin(9600);
  delay(4000);

  WiFi.begin(ssid, password);
 
  while (WiFi.status() != WL_CONNECTED) {
    delay(1000);
    Serial.println("Waiting for Wifi");
  }
 
  Serial.println("Wifi connected");
  delay(1000);
}



void loop()
{ 
  if ((WiFi.status() == WL_CONNECTED)) //Check the current connection status
  {  
    HTTPClient http;;

    bool httpInitResult = http.begin("http://192.168.0.105:485/esp32.html");

    http.addHeader("Content-Type", "text/html");

    if( httpInitResult == false )
    {
      Serial.println("http.begin() failed!");
    }
    else
    {
      int httpCode = http.GET();
    
      Serial.print("httpCode: ");
      Serial.println(httpCode);
     
      if (httpCode > 0) //Check for the returning code
      { 
        String payload = http.getString();
    
        Serial.print("Payload: ");
        Serial.println(payload);
      }
      else 
      {
        Serial.println("Error on HTTP request");
      }
    }
    Serial.println("Trace just before http.end()");
    http.end(); //Free the resources
    Serial.println("Trace just after http.end()");
  }
  delay(1000);
}

Output from Serial monitor with debug traces enabled:

15:36:22.978 -> Waiting for Wifi
15:36:22.978 -> Wifi connected
15:36:23.934 -> [V][HTTPClient.cpp:235] beginInternal(): url: http://192.168.0.105:485/esp32.html
15:36:24.034 -> [D][HTTPClient.cpp:276] beginInternal(): host: 192.168.0.105 port: 485 url: /esp32.html
15:36:24.135 -> [D][HTTPClient.cpp:1025] connect():  connected to 192.168.0.105:485
15:36:24.382 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'HTTP/1.1 200 OK'
15:36:24.471 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Content-Length: 148'
15:36:24.552 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Content-Type: text/html'
15:36:24.633 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Server: rpi'
15:36:24.715 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Date: Thu, 23 Jan 2020 14:36:23 GMT'
15:36:24.796 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Connection: close'
15:36:24.877 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: ''
15:36:24.959 -> [D][HTTPClient.cpp:1158] handleHeaderResponse(): code: 200
15:36:25.000 -> [D][HTTPClient.cpp:1161] handleHeaderResponse(): size: 148
15:36:25.081 -> [D][HTTPClient.cpp:1295] writeToStreamDataBlock(): connection closed or file end (written: 148).
15:36:25.163 -> [D][HTTPClient.cpp:370] disconnect(): tcp stop
15:36:25.246 -> 
15:36:25.246 -> Payload: <!doctype html><html lang="en"><head><meta charset="utf-8"><title>Esp32</title></head><body><p>2020-01-23 15:36:23;23.13;4.63;2.3;</p></body></html>
15:36:25.399 -> Trace just before http.end()
15:36:25.399 -> [D][HTTPClient.cpp:383] disconnect(): tcp is closed
15:36:25.500 -> 
15:36:26.350 -> [V][HTTPClient.cpp:235] beginInternal(): url: http://192.168.0.105:485/esp32.html
15:36:26.450 -> [D][HTTPClient.cpp:276] beginInternal(): host: 192.168.0.105 port: 485 url: /esp32.html
15:36:26.504 -> [D][HTTPClient.cpp:1025] connect():  connected to 192.168.0.105:485
15:36:26.851 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'HTTP/1.1 200 OK'
15:36:26.952 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Content-Length: 148'
15:36:27.005 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Content-Type: text/html'
15:36:27.106 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Server: rpi'
15:36:27.152 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Date: Thu, 23 Jan 2020 14:36:26 GMT'
15:36:27.253 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Connection: close'
15:36:27.353 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: ''
15:36:27.406 -> [D][HTTPClient.cpp:1158] handleHeaderResponse(): code: 200
15:36:27.453 -> [D][HTTPClient.cpp:1161] handleHeaderResponse(): size: 148
15:36:27.507 -> [D][HTTPClient.cpp:1295] writeToStreamDataBlock(): connection closed or file end (written: 148).
15:36:27.607 -> [D][HTTPClient.cpp:370] disconnect(): tcp stop
15:36:27.707 -> 
15:36:27.707 -> Payload: <!doctype html><html lang="en"><head><meta charset="utf-8"><title>Esp32</title></head><body><p>2020-01-23 15:36:25;23.13;4.63;2.3;</p></body></html>
15:36:27.839 -> Trace just before http.end()
15:36:27.886 -> Guru Meditation Error: Core  1 panic'ed (LoadProhibited). Exception was unhandled.
15:36:27.970 -> Core 1 register dump:
15:36:27.970 -> PC      : 0x4015d4d8  PS      : 0x00060430  A0      : 0x800d36db  A1      : 0x3ffb1e80  
15:36:28.109 -> A2      : 0x3ffb1f10  A3      : 0x00000002  A4      : 0x0000001c  A5      : 0x0000ff00  
15:36:28.155 -> A6      : 0x00ff0000  A7      : 0xff000000  A8      : 0x800d6591  A9      : 0x3ffb1e60  
15:36:28.256 -> A10     : 0x3ffbda34  A11     : 0x3f40240a  A12     : 0x00000002  A13     : 0x0000ff00  
15:36:28.356 -> A14     : 0x00ff0000  A15     : 0xff000000  SAR     : 0x00000004  EXCCAUSE: 0x0000001c  
15:36:28.456 -> EXCVADDR: 0x00000012  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xffffffff  
15:36:28.557 -> 
15:36:28.557 -> Backtrace: 0x4015d4d8:0x3ffb1e80 0x400d36d8:0x3ffb1ea0 0x400d37a9:0x3ffb1ec0 0x400d1ba6:0x3ffb1ee0 0x400d7775:0x3ffb1fb0 0x40088b9d:0x3ffb1fd0
15:36:28.710 -> 
15:36:28.710 -> Rebooting...
15:36:28.710 -> 	1⸮⸮⸮��⸮V�ŌHT⸮⸮Xl@�⸮[D][WiFiGeneric.cpp:337] _eventCallback(): Event: 0 - WIFI_READY
15:36:33.174 -> [D][WiFiGeneric.cpp:337] _eventCallback(): Event: 2 - STA_START
15:36:33.328 -> [D][WiFiGeneric.cpp:337] _eventCallback(): Event: 4 - STA_CONNECTED
15:36:33.629 -> [D][WiFiGeneric.cpp:337] _eventCallback(): Event: 7 - STA_GOT_IP
15:36:33.676 -> [D][WiFiGeneric.cpp:381] _eventCallback(): STA IP: 192.168.0.112, MASK: 255.255.255.0, GW: 192.168.0.1
15:36:34.130 -> Waiting for Wifi
15:36:34.130 -> Wifi connected
15:36:35.132 -> [V][HTTPClient.cpp:235] beginInternal(): url: http://192.168.0.105:485/esp32.html
15:36:35.232 -> [D][HTTPClient.cpp:276] beginInternal(): host: 192.168.0.105 port: 485 url: /esp32.html
15:36:37.038 -> [D][HTTPClient.cpp:1025] connect():  connected to 192.168.0.105:485
15:36:37.393 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'HTTP/1.1 200 OK'
15:36:37.493 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Content-Length: 148'
15:36:37.540 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Content-Type: text/html'
15:36:37.641 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Server: rpi'
15:36:37.694 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Date: Thu, 23 Jan 2020 14:36:36 GMT'
15:36:37.794 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: 'Connection: close'
15:36:37.895 -> [V][HTTPClient.cpp:1123] handleHeaderResponse(): RX: ''
15:36:37.941 -> [D][HTTPClient.cpp:1158] handleHeaderResponse(): code: 200
15:36:37.995 -> [D][HTTPClient.cpp:1161] handleHeaderResponse(): size: 148
15:36:38.095 -> [D][HTTPClient.cpp:1295] writeToStreamDataBlock(): connection closed or file end (written: 148).
15:36:38.164 -> [D][HTTPClient.cpp:370] disconnect(): tcp stop
15:36:38.233 -> 
15:36:38.233 -> Payload: <!doctype html><html lang="en"><head><meta charset="utf-8"><title>Esp32</title></head><body><p>2020-01-23 15:36:36;23.13;4.63;2.3;</p></body></html>
15:36:38.370 -> Trace just before http.end()
15:36:38.404 -> Guru Meditation Error: Core  1 panic'ed (IllegalInstruction). Exception was unhandled.
15:36:38.496 -> Core 1 register dump:
15:36:38.543 -> PC      : 0x54000000  PS      : 0x00060430  A0      : 0x800d36db  A1      : 0x3ffb1e80  
15:36:38.644 -> A2      : 0x3ffb1f10  A3      : 0x54000000  A4      : 0x0000001c  A5      : 0x0000ff00  
15:36:38.744 -> A6      : 0x00ff0000  A7      : 0xff000000  A8      : 0x8015d4de  A9      : 0x3ffb1e60  
15:36:38.797 -> A10     : 0x3ffbdaf0  A11     : 0x3f40240a  A12     : 0x00000002  A13     : 0x0000ff00  
15:36:38.898 -> A14     : 0x00ff0000  A15     : 0xff000000  SAR     : 0x00000004  EXCCAUSE: 0x00000000  
15:36:38.998 -> EXCVADDR: 0x00000000  LBEG    : 0x400014fd  LEND    : 0x4000150d  LCOUNT  : 0xffffffff  
15:36:39.098 -> 
15:36:39.098 -> Backtrace: 0x54000000:0x3ffb1e80 0x400d36d8:0x3ffb1ea0 0x400d37a9:0x3ffb1ec0 0x400d1ba6:0x3ffb1ee0 0x400d7775:0x3ffb1fb0 0x40088b9d:0x3ffb1fd0
15:36:39.246 -> 
15:36:39.246 -> Rebooting...
15:36:39.299 -> 	

Output from ESP Exception decoder:

Decoding stack results
0x4015d4d8: HTTPClient::connected() at C:\Users\kfj41\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\HTTPClient\src\HTTPClient.cpp line 395
0x400d36d8: HTTPClient::disconnect(bool) at C:\Users\kfj41\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\HTTPClient\src\HTTPClient.cpp line 359
0x400d37a9: HTTPClient::end() at C:\Users\kfj41\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\libraries\HTTPClient\src\HTTPClient.cpp line 347
0x400d1ba6: loop() at C:\Users\kfj41\Documents\Arduino\Display SPI test\TFT_eSPI_WiFi_Http_req_5 without display\TFT_eSPI_WiFi_Http_req_4/TFT_eSPI_WiFi_Http_req_4.ino line 60
0x400d7775: loopTask(void*) at C:\Users\kfj41\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.4\cores\esp32\main.cpp line 19
0x40088b9d: vPortTaskWrapper at /home/runner/work/esp32-arduino-lib-builder/esp32-arduino-lib-builder/esp-idf/components/freertos/port.c line 143

[lbernstone]

If you change the http.end to:
if (http.connected()) http.end();
do you get an exception?

[KenthJ]

if (http.connected()) http.end();

Ibernstone, I tried your suggestion, but unfortunately it didn't solve the problem.

Adding more traces shown that the ESP32 often crashes before the http.end() is called at all

Could the error be related to that I am using a fixed IP address instead of a domain name, and no DNS lookup is done?

[KenthJ]

Anybody able to help here?

I would really appriciate some help with this.

Thanks in advance

[liebman]

I can't reproduce this (with the URL changed to https://jigsaw.w3.org/HTTP/300/Overview.html)
Can you share how the server is configured so I can try to set that up here? (its a Raspberry Pi?)

[KenthJ]

Liebman, yes it is a Raspberry Pi running Windows 10 IoT

I inserted details below, do you need anything else?

Header from Fiddler
HTTP/1.1 200 Connection Established Content-Length: 148 Content-Type: text/html Server: rpi Date: Sun, 02 Feb 2020 15:54:59 GMT Connection: close

Raw content from Fiddler:
HTTP/1.1 200 Connection Established
Content-Length: 148
Content-Type: text/html
Server: rpi
Date: Sun, 02 Feb 2020 15:54:59 GMT
Connection: close
<!doctype html><title>Esp32</title>

2020-02-02 16:54:59;23.88;3.56;2.3;

[liebman]

OK, I won't be loading Windows IoT on my Pi. I'll set the web server on it up with something that looks similar-ish.

[KenthJ]

Great, I really appreciate that.

The raw content in my original post doesn't look correctly, since the html part isn't shown as I have written it, but I guess that is not so important, since I have validated the html content

[liebman]

I've been unable to duplicate the crash with current git version and apache2 configured on a Pi with: (/etc/apache2/sites-enabled/001-esp32.conf )

Listen 485
<VirtualHost *:485>
	ServerAdmin webmaster@localhost
	DocumentRoot /var/www/html
	ErrorLog ${APACHE_LOG_DIR}/error.log
	CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

# vim: syntax=apache ts=4 sw=4 sts=4 sr noet

and the html file: (/var/www/html/esp32.html )

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html lang="en">
  <head>
    <meta http-equiv="content-type" content="text/html; charset=utf-8">
    <title>esp32 test</title>
  </head>
  <body>
    This is a test.	
  </body>
</html>

[KenthJensen]

Liebman

Thank you so much for your support, I really appreciate it.

I have an extra Raspberry PI laying around, so I will try your example above on that one, it looks fairly simple.

Have you run it on Rasbbian on your Raspberry PI, and can I just install it via NOOBS?

[liebman]

I used Raspbian.

[KenthJ]

After a lot of trial and error I managed to get the Apache webserver up and running on Raspbian, and I agree, things worked fine using this webserver, the ESP32 didn't crash here.

I noticed that my old server used the following in the http header:
Connection: close

But the Apache webserver used:
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive

I tried using the same in the header from my original webserver, and voila, now the ESP32 didn't crash anymore.

This is a workaround I easily can live with permanently, but I guess it's something the httpCLient should be robust for, and not crash.

But thank you very much Liebman, so nice to get the problem solved.

[liebman]

I'll change the apache configuration I used to disable keep-alive and see if I can reproduce the issue.

[liebman]

Even with keep-alive disabled in apache I can't reproduce the crashes.

[Zenock]

Same issue. I'm not sure how to change the header from my server. :-( Mostly on http.end() but occasionally elsewhere.

Guru Meditation Error: Core 1 panic'ed (LoadProhibited). Exception was unhandled.
Core 1 register dump:
PC : 0x4015c69c PS : 0x00060630 A0 : 0x800d329a A1 : 0x3ffb1ef0
A2 : 0x3ffc106c A3 : 0x00000000 A4 : 0x00000004 A5 : 0x0000ff00
A6 : 0x00ff0000 A7 : 0xff000000 A8 : 0x800d50c1 A9 : 0x3ffb1ed0
A10 : 0x3ffba860 A11 : 0x3f4015f2 A12 : 0x00000002 A13 : 0x0000ff00
A14 : 0x00ff0000 A15 : 0xff000000 SAR : 0x0000001b EXCCAUSE: 0x0000001c
EXCVADDR: 0x00000010 LBEG : 0x400014fd LEND : 0x4000150d LCOUNT : 0xffffffff

Backtrace: 0x4015c69c:0x3ffb1ef0 0x400d3297:0x3ffb1f10 0x400d3309:0x3ffb1f30 0x400d1a32:0x3ffb1f50 0x400d1b5a:0x3ffb1f90 0x400d62bd:0x3ffb1fb0 0x40088b9d:0x3ffb1fd0

Rebooting...

I'm using a hosted web server. Not sure I can change the header. Wouldn't know how if I could.

[antonmeyer]

Looks like I have a similar Problem. but in my case it triggers InstrFetchProhibited.
the backtrace initial pointed to the display u8x8. but esp32 crashes also without display.