Not working with MPU-9250 IMU, reads fail

[MiloMindbender]

Hardware:

Board: Adafruit ESP32 feather
Core Installation/update date: 1.0.0
IDE name: Arduino IDE
Flash Frequency: 80Mhz
Upload Speed: 921600

Description:

I noticed in some closed issues there were problems with the MPU-9250 IMU that were apparently resolved, but it appears these problems may have come back.

My ESP32 is unable to communicate with the https://www.sparkfun.com/products/13762 9250 i2c breakout board. When I got this it seemed to be running fine but with the latest release of ESP32 arduino support this doesn't seem to work any more.

Samples from this repository
https://github.com/sparkfun/SparkFun_MPU-9250-DMP_Arduino_Library
or this one
https://github.com/sparkfun/SparkFun_MPU-9250_Breakout_Arduino_Library

Both fail in similar ways, the writes appear to go through and be acked and all the reads fail with a nak according to my Saleae logic analyzer. It is not even possible to retrieve the "who am I" value.

I have tried adding 3.3k pullups to the setup, but this does not make any difference.
I have tried using 100,000 and 400,000 speeds on Wire, no difference.

I have also double-checked the wires, only 4 (2 power and SDA/SCL, and they are correct. Everything is powered from the 3.3v regulator on the Adafruit ESP32 Feather Board.

On one of my computers at home these samples DO seem to work, but I think that machine might have an older release of the ESP32 arduino libraries, I will check that and report back soon.

If it would be helpful, I can post a dump of the I2C traffic from my logic analyzer (software to view it is free). If anyone wants it, let me know, I need to re-capture the data to shrink it's size a bit.

[stickbreaker]

@MiloMindbender there is a known problem with Release 1.0.0, a fix has been applied to the main branch. Until a new release has been compiled you can apply the fix by substituting the four i2c related files from the github source to your Arduino package location.

The Four files are:

• \libraries\Wire\src\Wire.h
• \libraries\Wire\src\Wire.cpp
• \cores\esp32\esp32-hal-i2c.h
• \cores\esp32\esp32-hal-i2c.c

under Windows 7 pro the Arduino Board Manager installs the package into:

C:\Users\{user}\AppData\Local\Arduino15\packages\esp32\hardware\esp32\1.0.0

Chuck.

[MiloMindbender]

I will give this a try. Last night I did regression testing and found the MPU-9250 worked with 1.0.0-rc3 but failed on anything newer than that.

Also, I've noticed the i2c frequency on ESP32 is off-spec. it's running 98khz when set to 100 and 382khz when set to 400.

[stickbreaker]

@MiloMindbender The ESP32 hardware uses clock cycle counting to control bus frequency. When it is sourcing SCL a single SCL cycle is constructed of four parts:

• first it pulls SCL Low
• second it detects when SCL is low and starts counting the pre-programmed CPU clock count for the low period.
• third it releases SCL (the pullup resistor and bus capacitance controls the time it takes SCL to reach the HIGH level).
• forth, it then detects the high level and starts counting the pre-programmed number of CPU clock cycles for the high period.

So, the bus capacitance and the pullup values distort the bus frequency from the pre-programmed values. Wire.setClock() can set the pre-programmed values from 10KHz to over 1.5MHz. If you need the bus frequency to be a specific period, you could use Wire.setClock() to adjust for you specific circuit. But, do realize that because of the delayed positive leading edge, your high/low periods will be asymmetric with a noticeably short HIGH component.

Chuck

[MiloMindbender]

I just replaced the 4 files, it works better but is intermittant. Some MPU9250 samples will start up ok the first time and maybe a time or two after pressing reset, then they start to fail. A full power off is required to get it working again.

Some of the samples such as MPU9250BasicAHRS_I2C still fail every time. These are trying to read the WHO_AM_I register and always fail at that point.

I tried inserting a 2 second delay after Wire.begin to give things a chance to stabalize, but this did not help.

[stickbreaker]

@MiloMindbender are you seeing any debug output?

[MiloMindbender]

The only output I get is the usual stuff when you press the reset button.

ets Jun 8 2016 00:22:57

rst:0x1 (POWERON_RESET),boot:0x13 (SPI_FAST_FLASH_BOOT)
configsip: 0, SPIWP:0xee
clk_drv:0x00,q_drv:0x00,d_drv:0x00,cs0_drv:0x00,hd_drv:0x00,wp_drv:0x00
mode:DIO, clock div:1
load:0x3fff0018,len:4
load:0x3fff001c,len:808
load:0x40078000,len:6084
load:0x40080000,len:6696
entry 0x400802e4

Do I need to set anything to get more debugging output?

[stickbreaker]

@MiloMindbender set the Core Debug Level to at least DEBUG and run you code, if the i2c system has any problems it will vomit out what it detects.

Also, inside \cores\esp32\esp32-hal-i2c.c there is a #define that will increase debug support and verbosity of messages;
line 45 here

Chuck.

[MiloMindbender]

With just the DEBUG set and the line 45 mod I am getting nothing when it works. After pressing reset a few times I got it to fail and got this:

[W][esp32-hal-i2c.c:1363] i2cCheckLineState(): invalid state sda=0, scl=1

[E][esp32-hal-i2c.c:1379] i2cCheckLineState(): Bus Invalid State, TwoWire() Can't init

Logic analyzer shows sda low, scl high also. Even when I press/hold the reset button they remain in this state.

Note this is with 10k pullups on the mpu 9250 board plus an additional 3.3k added externally.

[MiloMindbender]

More info. I removed the 3.3k pullups and now it takes 6-7 tries to get a freeze.

I also tried clipping the logic analyzer to the MPU-9250 pins, getting the setup into a frozen state and then disconnecting SDA/SCL from the ESP, this showed SDA low and SCL high also. So it appears the 9250 has gotten into a state where it's pulling SDA low for some reason.

Since the ESP can't communicate with it, a software reset of the 9250 isn't possible, so it remains stuck this way till you power off...not sure why this happens.

[stickbreaker]

@MiloMindbender you can use Wire.begin() as a bus reset function. The newest Github with PR#1717 changed Wire.begin() to a function to will cycle the bus if necessary.

If you can detect the lockup, try:

if(lockedUP){
  uint32_t timeout =millis();
  while((millis()-timeout<1000)&&(!Wire.begin())){ // twiddle thumb
   delay(5); // maybe IMU is busy? give it a chance to finish
  }
}
If(!Wire.begin()){ 
   Serial.println(" I2C bus did not Init, Fatal")
} else {
  // bus is good, back to regularly scheduled programming.
}

Chuck.

[MiloMindbender]

Not sure if this is it or not...In the 9250 datasheet they mention

"Each byte transferred must be followed by an acknowledge bit. To acknowledge, the slave device pulls the SDA line LOW and keeps it LOW for the high period of the SCL line. "

When it fails on a reset of the ESP32 it appears that SCL is high and SDA has just gone low and it stays this way. It is doing a "setup read". So maybe the 9250 was in the middle of an ACK when reset was pushed and is waiting for SCL to go low, but it never does?

Across a reset, should the SCL/SDA lines go low and back to high or just stay high?

Not an expert, just guessing.

[stickbreaker]

@MiloMindbender The SDA and SCL lines are only pulled high by the pullup resistors, The bus is a Wired 'AND' configuration. To get a HIGH, all devices on the bus must stop pulling the Signal Low before it can go High.

The I2C protocol has a defined method to recover from a SDA low lockup. SDA is released by the Master and 9 SCL clock cycles are pulsed on the bus. All I2C slave devices must release SDA when they receive this sequence.

I just had a thought; maybe the IMU is out of bit sequence. it thinks it is working on bit 4 and the ESP32 thinks it is the ACK bit.

Try this change to`esp32-hal-i2c.c' in the i2cCheckLineState function
Original code:

  if(!digitalRead(sda) || !digitalRead(scl)) { // bus in busy state
        log_w("invalid state sda=%d, scl=%d\n", digitalRead(sda), digitalRead(scl));
        digitalWrite(sda, HIGH);
        digitalWrite(scl, HIGH);
        delayMicroseconds(5);
        digitalWrite(sda, LOW);
        for(uint8_t a=0; a<9; a++) {
            delayMicroseconds(5);
            digitalWrite(scl, LOW);
            delayMicroseconds(5);
            digitalWrite(scl, HIGH);
        }
        delayMicroseconds(5);
        digitalWrite(sda, HIGH);
    }

Changed code:

  if(!digitalRead(sda) || !digitalRead(scl)) { // bus in busy state
        log_w("invalid state sda=%d, scl=%d\n", digitalRead(sda), digitalRead(scl));
        digitalWrite(sda, HIGH);
        digitalWrite(scl, HIGH);
        delayMicroseconds(5);
        digitalWrite(sda, LOW);
        for(uint8_t a=0; a<9; a++) {
             delayMicroseconds(5);
            digitalWrite(scl, LOW);
            delayMicroseconds(5);
            digitalWrite(scl, HIGH);
            delayMicroSeconds(2);
            digitalWrite(sda,HIGH); // simulate STOP
            if(digitalRead(sda)){ // bus recovered, all done
                break;
            }
        }
        delayMicroseconds(5);
        digitalWrite(sda, HIGH);
    }

Chuck.

[MiloMindbender]

Will try to get your test code in shortly. In the meantime maybe this logic analyzer trace is helpful?

This shows the bus working ok, then reset is pressed and it in the low/high state forever.

Hope I explained myself correctly earlier, what I meant was that maybe when the ESP32 was reset the 9250 was working on an ACK and had just pulled SDA low, but because of the reset SCL is now high. the 9250 is waiting on an SCL low pulse that never comes, so it just stays stuck in that state?

Like I said, not an expert, does the logic analyzer trace help?

[MiloMindbender]

Just to be totally clear, this lockup only happens when reset is pressed and you often have to reset 7 or more times before it ends up in this state. The I2C does not appear to lock up at any other time. I'm letting one of the test programs run for a while to be sure.

Also to be totally clear, in the logic analyzer trace above, where the signal stops is where reset was pressed and there is no activity on I2C once the ESP32 finishes its reset cycle.

Also, just checking, would you recommend I pull down a zip of the latest code here and install it before putting in and testing your patch above?