Skip to content

a lot of unexpected EOF error when reading from partition #1286

Open
@acynothia

Description

@acynothia

Describe the bug

A clear and concise description of what the bug is.

a lot of unexpected EOF error when reading from partition cause here.

kafka-go/reader.go

Lines 1421 to 1433 in 2af3101

default:
var kafkaError Error
if errors.As(err, &kafkaError) {
r.sendError(ctx, err)
} else {
r.withErrorLogger(func(log Logger) {
log.Printf("the kafka reader got an unknown error reading partition %d of %s at offset %d: %s", r.partition, r.topic, toHumanOffset(offset), err)
})
r.stats.errors.observe(1)
conn.Close()
break readLoop
}
}

I was try to log the detail found that when highWaterMark + 1 == offset and remain is 0. newMessageSetReader will return io.EOF(because remain is 0, read head first offset with io.EOF error)

kafka-go/conn.go

Lines 859 to 866 in 2af3101

var msgs *messageSetReader
if err == nil {
if highWaterMark == offset {
msgs = &messageSetReader{empty: true}
} else {
msgs, err = newMessageSetReader(&c.rbuf, remain)
}
}

It's cause a lot of error log and conn.Close a lot of recreate connection and TIME_WAIT.

Kafka Version

  • What version(s) of Kafka are you testing against?
    v2.2.0
  • What version of kafka-go are you using?
    v0.4.47

To Reproduce

Resources to reproduce the behavior:

---
# docker-compose.yaml
#
# Adding a docker-compose file will help the maintainers setup the environment
# to reproduce the issue.
#
# If one the docker-compose files available in the repository may be used,
# mentioning it is also a useful alternative.
...
package main

import (
    "github.com/segmentio/kafka-go"
)

func main() {
    // Adding a fully reproducible example will help maintainers provide
    // assistance to debug the issues.
    ...
}

Expected Behavior

A clear and concise description of what you expected to happen.

Don't return error when highWaterMark + 1 == offset and remain is 0.

Shall we just add remain == 0 condition ?

if highWaterMark == offset {

  	if highWaterMark == offset || remain == 0 { 

IMO, when remain is 0 return empty is reasonable.

Observed Behavior

A clear and concise description of the behavior you observed.

Often times, pasting the logging output from a kafka.Reader or kafka.Writer will
provide useful details to help maintainers investigate the issue and provide a
fix. If possible, providing stack traces or CPU/memory profiles may also contain
valuable information to understand the conditions that triggered the issue.

Additional Context

Add any other context about the problem here.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions