Skip to content

Add the streaming indexer to the bindings #192

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 2 commits into from

Conversation

carlosmn
Copy link
Member

There are a few things that I'm not so sure about, style-wise.

Firstly, the indexer's stats are actually unsigned int in C, but if I try to put that in the struct, I get a warning that it's not CLS complaint, so it it may not work. @nulltoken, do you have any insights into this?

Secondly, I transform the stats into the C# variant after each add, which could be too much work, if the stats are looked up less often. I thought about doing it on demand, but that would mean locking and I worry that it would mean even more work.

Thirdly, is there some extablished convention whereby I can make clear that you shouldn't call Indexer.Hash() before the indexer is done?

@carlosmn
Copy link
Member Author

I've left Add(byte[]) for now until we figure out whether I should just start a new Thread/Task when demultiplexing the input from the remote or we can find a better way of making it work.

@carlosmn
Copy link
Member Author

After much dragging my ass, I've updated this so it reads from a Stream instead of manually appending arrays. I think this is what we were talking about way back when, @nulltoken . Maybe I should change the author date to something more current...

@nulltoken
Copy link
Member

Haven't we previously discussed about also making the progress callback based? Unless I'm wrong, current design would require a separate thread to observe the progress.

@nulltoken
Copy link
Member

BTW, Travis doesn't like you today 😉

/home/travis/build/libgit2/libgit2sharp/CI-build.msbuild: error : LibGit2Sharp.Tests.IndexerFixture.CanIndexFromAStream: LibGit2Sharp.LibGit2SharpException : An error was raised by libgit2. Category = Invalid (Error).
Failed to open packfile.: Permission denied
/home/travis/build/libgit2/libgit2sharp/CI-build.msbuild: error :   at LibGit2Sharp.Core.Ensure.HandleError (Int32 result) [0x00000] in <filename unknown>:0 
  at LibGit2Sharp.Core.Ensure.ZeroResult (Int32 result) [0x00000] in <filename unknown>:0 
  at LibGit2Sharp.Core.Proxy.git_indexer_append (LibGit2Sharp.Core.Handles.IndexerSafeHandle handle, System.Byte[] data, UIntPtr length, LibGit2Sharp.Core.GitTransferProgress& progress) [0x00000] in <filename unknown>:0 

@carlosmn
Copy link
Member Author

We might have talked about that, yeah. The indexing is really something that happens in the background, but sure, I'll add some callback.

I'm not sure what to make of the travis error. I'll take a look tomorrow.


namespace LibGit2Sharp.Advanced
{
public class Indexer
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please add some doc explaining how by just using this type, the user will magically be teleported into a wonderful land full of rainbows and unicorns?

@carlosmn
Copy link
Member Author

I'm not sure wtf I was doing earlier, but the indexer has all these fancy parameters I was just skipping over.

I'm now facing adding the progress callback and I notice that RemoteCallbacks has this nice GitDownloadTransferProgressHandler that's private but I would very much like to use. Now, should I make it internal instead of private, or should we do some larger refactoring?

@jamill
Copy link
Member

jamill commented Dec 11, 2013

GitDownloadTransferProgressHandler depends on some internal state of RemoteCallbacks (the consumer supplied DownloadTransferProgress delegate to call, if any), so this will require more than just changing the accessibility of the method.

@carlosmn
Copy link
Member Author

Yeah, you're right. I guess I'll have to end up transplanting the logic so it uses local functions.

This allows the user to index a packfile they have in a file or are
retrieving through a Stream.
Since you cannot re-use the same Indexer object to index two different
files, it doesn't make much sense to provide a class which would allow
you to do this.

Instead expose two static methods which internally create an Indexer
instance and feed it from the stream or file.
@carlosmn
Copy link
Member Author

I've brought this back from the dead, the style issues have been addressed and we only actually deal with Streams anymore.

I'm not sure how much explanation there should be for the class; I figured mentioning it's the equivalent to index-pack should be enough for anybody who would want to use it.


do
{
read = stream.Read(buffer, 0, buffer.Length);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stream.Read can return negative here on failure - we should break if read <= 0.

@ethomson
Copy link
Member

Let's get this out of LibGit2Sharp.Advanced. Instead, let's just put it into LibGit2Sharp... This seems to be in keeping with the trend of making LibGit2Sharp the "core" area and move the "simpler" stuff into Commands...?

@bording bording closed this Apr 21, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants