Local test data

[gondzo]

It would be really helpful to have test data for project templates/phases/products that is identical to what we have in dev env. Right now it is really complex to create projects in local environment. Postman calls aren't helpful either.

What we need here is a simple script that will create the test data and update to the README.
Do we have something like this @RishiRajSahu @vikasrohit ?

[vikasrohit]

I don't think we have that yet, it would definitely be a good asset to have such script for future work. @gondzo do you think you would be able to get it done by copying a sample out of current dev templates?

[gondzo]

Yup, current dev templates would be really helpful.
Since we have migrations already configured, we can just add another migration that sets up test data.
Can you make a backup of dev db and we can create the migration script in a private task ?
This should also make it easy to sync dev and prod templates without manually adding them via postman calls.

[vikasrohit]

Since we have migrations already configured, we can just add another migration that sets up test data.

I think we should keep the test data script separate from migration scripts because migration scripts are intended for production data.

Can you make a backup of dev db and we can create the migration script in a private task ?

Would the backup of only the templates table work?

This should also make it easy to sync dev and prod templates without manually adding them via postman calls.

I think it would be risky to sync templates from dev to production in any case. I think it should always be manual to upload the templates on production, IMO.

[gondzo]

OK, we can work with backups for template tables only, we don't need project tables and the script would be separate from the migrations script

[gondzo]

@vikasrohit can we start some work on this, it's very hard to have the community working on project service without sample data.
Besides sample data we also need to:

1. update docs (readme is heavily outdated). maybe adding an architecture overview doc or diagram would be helpful - it's not easy to figure out how we're using notifications&Kafka, rabbitMQ, elasticsearch, dynamo, etc
2. update local services data - we should agree on a set of real user accounts and add them as mocks in local services (docker) - trying to use local environment while integrating with connect app is nearly impossible if project service can't load the same members as connect app.

As it is now, we aren't getting high quality submissions from the challenges, and we can't use entire review pool to review the changes - many points will be missed if a reviewer doesn't have prior experience with this codebase.

FYI @RishiRajSahu

[RishiRajSahu]

Hey @gondzo, i see you created a PR #248 and it seems you are in context, so would you like to again take over this issue ?

[gondzo]

sure. #248 solves most of the requirements.
the only issue is that metadata in dev can't be inserted directly in local database - probably there is some stale data in dev db so validations fail sometimes.

[RishiRajSahu]

@gondzo Could we add a check to remove that stale data and empty the dev db before this push of metadata ? It would be great to make the local deployment as smooth as possible.