Index projects even if cannot get member details

[maxceem]

if we are not able to fetch member details for the active members, it does not index that project:

• tc-project-service/src/routes/admin/project-index-create.js

  Line 87 in 496e4ac

  return null;

• tc-project-service/src/routes/admin/project-index-create.js

  Line 92 in 496e4ac

  return null;

We should still index the project even if we cannot get member details for project members.

[vikasrohit]

@maxceem I think this one still holds true but we need to update the links in the issue to point to correct line number because code seems to be refactored. And if would be great if we can get it done in May release.

[maxceem]

The code has been updated and moved to another file.

If we didn't get user details from the Member Service, we should log error, but the project should be still added to the ES https://github.com/topcoder-platform/tc-project-service/blob/develop/src/utils/es.js#L168-L172.

[vikasrohit]

This is about admin endpoint, right @maxceem? For admin endpoint, it might make sense to error out if we are not able to find member details, because we use this endpoint when we want to fix some issue with the data e.g. one of the project member's first name, last name or email is not correct or infact was not indexed and it is causing the project search to not return that project when search for that user. So, if this endpoint just indexes it again without member details, we might think it should be fixed after calling this endpoint.
However, on the other side, I don't see any good solution for handling this situation. I think may be a separate admin endpoint to fix members of the project, might help us in this direction because that endpoint can fail safely when it does not get required data from member api.

[maxceem]

Yes, it's for admin endpoint. The main idea is to avoid situations when some project is not indexed at all because of user details when we index 1000 projects in batches.

To solve the issue you've raised in the comment above, I suggest to improve the output of admin endpoint response. It should return the log in the response see details #562.

So in case we are just reindexing a single project to fix member, we would see if there were any errors or no in the response.

[vikasrohit]

Streaming is something I am not 100% sure as of now. But I think we can go with the fixing the original problem mentioned in the issue.