Esptool no_reset option

[mribble]

Basic Infos

Hardware

Hardware: ESP-12E
Core Version: 2.4.0-rc1

Description

I'm programming the ESP8266 through another processor (specifically the sam3x -- used in Arduino Due). I'm using the USB Native port on the sam3x. The win10 usb->serial driver gets confused with how esptool treats the com port. To fix this I need to add the following command lines to esptool.py: "--before no_reset --after no_reset". It would be nice if I could select this through the arduino IDE.

I noticed you are using esptool v0.4.12. The latest stable is v2.1. This new version has a lot of improvements. One of the biggest ones that would impact many users is it compresses uploads and empty SPIFFS uploads compress really well. I saw uploads with bigs SPIFFS go from what seemed like minutes to just a few seconds.

1. Has anyone considered upgrading to a new version of the esptool?
2. Would it be possible to add a new board called "Generic Esp8266 Module (no reset)"?

I might be willing to work on such changes if the problem is getting someone to do the development work, but since I haven't committed any code to this project I'd like to hear back from the senior developers if such commits would be accepted (assuming good quality code of course).

Settings in IDE

Module: Generic ESP8266 Module
Flash Size: 4MB
CPU Frequency: 80Mhz
Flash Mode: qio
Flash Frequency: 40Mhz
Upload Using: Special
Reset Method: Special

Sketch

NA

Debug Messages

NA

[igrr]

Please note that there are two esptools: esptool.py ( https://github.com/espressif/esptool) and esptool-ck ( https://github.com/igrr/esptool-ck), which are unrelated, thus the version difference. It is possible to use esptool.py in this project, but requires work to be done to package esptool.py, pyserial, and Python runtime into a downloadable package for each OS, so that we don't have to depend on presence of Python and pyserial on the user's computer. Back to the original question, esptool-ck has the equivalent option to avoid doing the reset, called '-cb none'. It can be exposed in the IDE menu by adding a line to boards.txt.

…

On Wed, Nov 8, 2017, 05:37 Maurice Ribble ***@***.***> wrote: Basic Infos Hardware Hardware: ESP-12E Core Version: 2.4.0-rc1 Description I'm programming the ESP8266 through another processor (specifically the sam3x -- used in Arduino Due). I'm using the USB Native port on the sam3x. The win10 usb->serial driver gets confused with how esptool treats the com port. To fix this I need to add the following command lines to esptool.py: "--before no_reset --after no_reset". It would be nice if I could select this through the arduino IDE. I noticed you are using esptool v0.4.12. The latest stable is v2.1. This new version has a lot of improvements. One of the biggest ones that would impact many users is it compresses uploads and empty SPIFFS uploads compress really well. I saw uploads with bigs SPIFFS go from what seemed like minutes to just a few seconds. 1. Has anyone considered upgrading to a new version of the esptool? 2. Would it be possible to add a new board called "Generic Esp8266 Module (no reset)"? I might be willing to work on such changes if the problem is getting someone to do the development work, but since I haven't committed any code to this project I'd like to hear back from the senior developers if such commits would be accepted (assuming good quality code of course). Settings in IDE Module: Generic ESP8266 Module Flash Size: 4MB CPU Frequency: 80Mhz Flash Mode: qio Flash Frequency: 40Mhz Upload Using: *Special* Reset Method: *Special* Sketch NA Debug Messages NA — You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub <#3809>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEJcelJnlEq3gGgDGlMti_7ym8QEN5Vaks5s0M2wgaJpZM4QVfQK> .

[d-a-v]

@mribble there is a PR for a boards.txt generator (#3722 #3582) where we can updates the menus.
About menus, is it sufficient to add a new item 'none' in the 'reset method' entry ?

[mribble]

@igrr thanks for setting me on the correct path. I didn't realize there were two different esptool projects. That clarifies a lot of my misunderstandings.

@d-a-v you are correct that adding a reset method of "none" seems like a better solution. I tried putting that in boards.txt manually with the following lines:
generic.menu.ResetMethod.none=none
generic.menu.ResetMethod.none.upload.resetmethod=none

That did update the UI in a reasonable way, and it add "-cd none" to the tool's command line as expected. However, my board didn't program (I just get the sync error as I was before). Do you guys know of what might be going wrong? If not I will debug the issue later today by digging into the esptool-ck. The win10 usb driver is kind of sensitive to oddities and I debugged esptool.py with a logic analyzer already so I'm happy to do the same for esptool-ck if needed.

@d-a-v if you think I should pull in your bigger board.txt generator pull request I can do that. But I was just trying to verify esptool-ck first. I do think this is a good option expose once I figure out why it's not working. I saw several people trying to do what I'm doing on the web and nobody else I found could get it working. Probably because they didn't spend as much time looking into it as I did, since it is certainly not due to my vast knowledge in this area. I'm just learning as I go so any suggestions you experts can make is appreciated as it will save me lots of time.

[igrr]

However, my board didn't program (I just get the sync error as I was before). Do you guys know of what might be going wrong?

Esptool resets the board in order to put it into bootloader mode. You have mentioned that your setup has an intermediate MCU. If you set -cd none, esptool will not use DTR or RTS signals to control GPIO0 and RST pins of the ESP8266. So your MCU needs to put the ESP into bootloader mode somehow before uploading the code (i.e. pull RST low, pull GPIO0 low, release RST, wait some time, release GPIO0). You can find the code used by esptool to reset the boards here:
https://github.com/igrr/esptool-ck/blob/master/espcomm/espcomm_boards.c#L65

[mribble]

Esptool resets the board in order to put it into bootloader mode

I am handling the bootloader reset on my microcontroller. I'm sure the reset code and wiring is all correct since I can upload fine using the esptool.py as long as I set those no_reset parameters I mentioned in the first post.

[mribble]

Ok, I think I found the problem in esptool-ck, but I haven't setup the environment needed to compile and test it. Also I'm not exactly sure how you'd want to implement the fix.

The problem is that esptool-ck isn't setting dtr lines correctly for Arduino Due's native usb serial port. This describes the problem: https://mpuprojectblog.wordpress.com/2013/07/19/arduino-dues-serialusb-and-processing/

I did confirm that the hack suggested in the link above of removing "if (_usbLineInfo.lineState > 0)" from the core arduino due library makes it so I can upload code using esptool-ck.

I think the correct fix is something that looks like esptool.py has already done. I'm not sure if open source licensing is compatible between these projects so instead of posting their code (you can find it in their esptool.py if you want by searching for DTR) let me post some pseudo code for the common dtr reset logic that is missing from esptool.ck:

  setDTR(LOW);
  setRTS(HIGH);
  delay(100ms);
  setDTR(HIGH);
  setRTS(LOW);
  delay(50ms);
  setDTR(LOW);

Looking through espcomm_sync in esptool.ck I didn't see anything like that. I think one place to put such code would be in espcomm_boards.c. Replace the null pointer for "none" in s_boards[] with a function to do this. That should solve my issue. If you think there is a better place to put such code let me know.

I don't know how to justify this change by saying it is the right thing to do for correct serial protocol since I'm not familiar enough with these standards or best practices. But it seems like anything that can communicate to Arduino Due's native USB serial seems to have that code. That is a lot of things such as popular terminal apps like putty. Hopefully that is enough justification for a change like this. If you need something more to justify this kind of change let me know what you are looking for.

This part of this bug belongs over on the esptool.ck project, but since @igrr owns that project and he's already looking at this bug I decided to put this update here. If you want to take this part of the bug over there that's fine. However, this bug should stay open to help track adding the "None" option to the boards.txt generator code you are working on.

[igrr]

The pseudocode you have posted looks about the same as board_nodemcu_rb function in esptool-ck, except that some delays are different. If your intent is to actually toggle RTS and DTR pins in a certain way, then this certainly doesn't qualify as "none" reset method. Perhaps you could try the existing "nodemcu" reset method first and see if it works for you?

…

On Wed, Nov 8, 2017, 22:46 Maurice Ribble ***@***.***> wrote: Ok, I think I found the problem in esptool-ck, but I haven't setup the environment needed to compile and test it. Also I'm not exactly sure how you'd want to implement the fix. The problem is that esptool-ck isn't setting dtr lines correctly for Arduino Due's native usb serial port. This describes the problem: https://mpuprojectblog.wordpress.com/2013/07/19/arduino-dues-serialusb-and-processing/ I did confirm that the hack suggested in the link above of removing "if (_usbLineInfo.lineState > 0)" from the core arduino due library makes it so I can upload code using esptool-ck. I think the correct fix is something that looks like esptool.py has already done. I'm not sure if open source licensing is compatible between these projects so instead of posting their code (you can find it in their esptool.py if you want by searching for DTR) let me post some pseudo code for the common dtr reset logic that is missing from esptool.ck: setDTR(LOW); setRTS(HIGH); delay(100ms); setDTR(HIGH); setRTS(LOW); delay(50ms); setDTR(LOW); Looking through espcomm_sync in esptool.ck I didn't see anything like that. I think one place to put such code would be in espcomm_boards.c. Replace the null pointer for "none" in s_boards[] with a function to do this. That should solve my issue. If you think there is a better place to put such code let me know. I don't know how to justify this change by saying it is the right thing to do for correct serial protocol since I'm not familiar enough with these standards or best practices. But it seems like anything that can communicate to Arduino Due's native USB serial seems to have that code. That is a lot of things such as popular terminal apps like putty. Hopefully that is enough justification for a change like this. If you need something more to justify this kind of change let me know what you are looking for. This part of this bug belongs over on the esptool.ck project, but since @igrr <https://github.com/igrr> owns that project and he's already looking at this bug I decided to put this update here. If you want to take this part of the bug over there that's fine. However, this bug should stay open to help track adding the "None" option to the boards.txt generator code you are working on. — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#3809 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AEJcevYZ42aD-1bJY-95tEDW-sFzYqkpks5s0b61gaJpZM4QVfQK> .

[mribble]

You are correct. I misread what was happening with RTS and DTR pins. esptool.py is not toggling them like I said for mode=none. I inverted condition when I was reading the code.

The problem is somewhere else. But that hack I mentioned before in arduino core did make things work so it's something in this area with the initial setup of the serial port.

My next guess is that in serialport.c maybe changing "sDCB.fDtrControl = DTR_CONTROL_DISABLE;" to enable would fix it. My thinking goes that the default value for DTR is supposed to be high voltage (or so I've read -- didn't verify). It seems I don't need to toggle the dtr pin, but perhaps it needs to be high. Do you know if changing any flags in that initialization structure would change the DTR value from low to high?

I'm happy to setup and test whatever solution we come up with, but I'd like to come up with a theory that we both agree has a chance at working first. However, if I'm taking up too much of your time I can go and experiment first and just come back when I'm more sure of a solution. Thanks for all your help!

Edit - Just to confirm since you asked the question, I tried using nodemcu and it did not work.

[mribble]

I filed a bug with more details to track down the esptool issue here.

The only issue this bug is now tracking is adding "none" as a reset method to the board.txt file.

[igrr]

As mentioned in the linked esptool-ck issue, boards.local.txt can be used to add custom menu items.

For example, create boards.local.txt file in the ESP8266 core directory (same directory as boards.txt) with the following contents:

generic.menu.ResetMethod.dtrset=dtrset
generic.menu.ResetMethod.dtrset.upload.resetmethod=dtrset

Then restart the IDE. dtrset option will appear in Tools > Reset method menu, once you select "Generic ESP8266 Module" in Tools > Board menu.