Description
IRC conversation between @diasdavid and @dignifiedquire
dignifiedquire> daviddias: I just realized that the streaming interface for the blockstore is a bit trickier than I hoped
we are passing instances of Block to it, to be written to disk
but that notion itself relies on having all the data in memory, in the associated buffer
<•daviddias> right, because you need the hash of it
so the question is can we make this a streaming interface all the way actually, and if so how
do you know if go has a streaming interface for the blockstoreß
daviddias: looks like go-ipfs is using in memory blocks only as well: https://github.com/ipfs/go-ipfs/blob/master/blocks/blocks.go#L25
<•daviddias> having streaming to disk
<•daviddias> is good because it is a way to capture the signal from the storage driver for backpressure
<•daviddias> but yeah, block-service has to be smart enough
<•daviddias> to capture those signals
<•daviddias> the streaming interface for reads is also important
<•daviddias> because we can pipe it to the network
<•daviddias> and even for writting, if we request a block
<•daviddias> we can pipe it to disk
<•daviddias> and then validate the hash with hashpipe
hmm
this will require a major refactor of a lot of parts :/
I think we should delay this
otherwise we will stretch this move to pull streams very long
daviddias: got everything passing with the current interface
<•daviddias> I see you point
<•daviddias> As always, please document in a issue
<•daviddias> So that we know when to come back to it :)