Skip to content

Commit 5dec625

Browse files
ThomasKranitsasThomasKranitsas
committed
Update readme
1 parent 2195009 commit 5dec625

File tree

4 files changed

+160
-50
lines changed

4 files changed

+160
-50
lines changed

‎README.md

Lines changed: 59 additions & 50 deletions
Original file line numberDiff line numberDiff line change
@@ -1,9 +1,28 @@
11
# Topcoder Challenge Resources Processor
22

3-
## Dependencies
3+
This service is used for processing kafka events related to challenge creation and create the default resources based on the members who have access to the same project.
44

5-
- nodejs https://nodejs.org/en/ (v10+)
6-
- Kafka
5+
### Development deployment status
6+
[![CircleCI](https://circleci.com/gh/topcoder-platform/resource-processor/tree/develop.svg?style=svg)](https://circleci.com/gh/topcoder-platform/resource-processor/tree/develop)
7+
8+
### Production deployment status
9+
[![CircleCI](https://circleci.com/gh/topcoder-platform/resource-processor/tree/master.svg?style=svg)](https://circleci.com/gh/topcoder-platform/resource-processor/tree/master)
10+
11+
## Intended use
12+
- Processor
13+
14+
## Related repos
15+
16+
- [Resources API](https://github.com/topcoder-platform/resources-api)
17+
- [Projects API](https://github.com/topcoder-platform/projects-api)
18+
- [Member API](https://github.com/appirio-tech/ap-member-microservice)
19+
20+
## Prerequisites
21+
22+
- [NodeJS](https://nodejs.org/en/) (v10+)
23+
- [Kafka](https://kafka.apache.org/)
24+
- [Docker](https://www.docker.com/)
25+
- [Docker Compose](https://docs.docker.com/compose/)
726

827
## Configuration
928

@@ -48,7 +67,12 @@ Also note that there is a `/health` endpoint that checks for the health of the a
4867
Configuration for the tests is at `config/test.js`, only add such new configurations if different than `config/default.js`:
4968
- WAIT_TIME: wait time used in test, default is 2000 or 2 seconds
5069

51-
## Local Kafka setup
70+
## Local Deployment
71+
72+
### Foreman Setup
73+
To install foreman follow this [link](https://theforeman.org/manuals/1.24/#3.InstallingForeman)
74+
To know how to use foreman follow this [link](https://theforeman.org/manuals/1.24/#2.Quickstart)
75+
### Local Kafka setup
5276

5377
- `http://kafka.apache.org/quickstart` contains details to setup and manage Kafka server,
5478
below provides details to setup Kafka server in Mac, Windows will use bat commands in bin/windows instead
@@ -73,67 +97,52 @@ Configuration for the tests is at `config/test.js`, only add such new configurat
7397
`bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic challenge.notification.create --from-beginning`
7498

7599

76-
## Local deployment
100+
### Local deployment
77101

78102
- start mock-api, go to `mock-api` folder, run `npm i` and `npm start`, mock api is running at `http://localhost:4000`
79103
- go to project root folder, install dependencies `npm i`
80104
- run code lint check `npm run lint`, running `npm run lint:fix` can fix some lint errors
81105
- start processor app `npm start`
82106

83107

84-
## Unit Tests and E2E Tests
85-
86-
Before running tests, setup and start kafka server, start the mock API, but do not start the processor app.
87-
88-
- run `npm run test` to execute unit tests.
89-
- run `npm run test:cov` to execute unit tests and generate coverage report.
90-
- run `npm run e2e` to execute e2e tests.
91-
- run `npm run e2e:cov` to execute e2e tests and generate coverage report.
108+
## Production deployment
109+
- TBD
92110

111+
## Running tests
93112

94-
## Verification
113+
### Configuration
114+
Test configuration is at `config/test.js`.
115+
The following test parameters can be set in config file or in env variables:
116+
- WAIT_TIME: wait time used in test
95117

96-
- setup and start kafka server, start mock API, start processor app
97-
- start kafka-console-producer to write messages to `challenge.notification.create` topic:
98-
`bin/kafka-console-producer.sh --broker-list localhost:9092 --topic challenge.notification.create`
99-
- write message:
100-
`{ "topic": "challenge.notification.create", "originator": "challenge-api", "timestamp": "2019-02-16T00:00:00", "mime-type": "application/json", "payload": { "id": "5505f779-b28b-428f-888b-e523b443f3ea", "typeId": "7705f779-b28b-428f-888b-e523b443f3ea", "track": "code", "name": "test", "description": "desc", "timelineTemplateId": "8805f779-b28b-428f-888b-e523b443f3ea", "phases": [{ "id": "8805f779-b28b-428f-888b-e523b443f3eb", "name": "phase", "isActive": true, "duration": 1000 }], "prizeSets": [{ "type": "Challenge prizes", "prizes": [{ "type": "1st", "value": 600 }] }], "reviewType": "community", "tags": ["tag1"], "projectId": 30055214, "startDate": "2019-02-19T00:00:00", "status": "Draft", "created": "2019-02-16T00:00:00", "createdBy": "tester" } }`
118+
### Prepare
119+
Before running tests, setup and start kafka server, start the mock API, but do not start the processor app.
120+
Various config parameters should be properly set.
101121

102-
- you will see app logging:
122+
### Running unit tests
123+
To run unit tests alone
103124
```bash
104-
info: Process message of challenge id 5505f779-b28b-428f-888b-e523b443f3ea and project id 30055214
105-
info: Found member ids [40152933, 40141336] of project id 30055214
106-
info: Created resource: {
107-
"challengeId": "5505f779-b28b-428f-888b-e523b443f3ea",
108-
"memberHandle": "lordofparadox",
109-
"roleId": "6605f779-b28b-428f-888b-e523b443f3ea",
110-
"id": "d4ac5715-b1ed-430b-86ca-56d022c97ce9",
111-
"memberId": "9e2ad182-d4f7-4c4d-9421-29bc8bd56dfb",
112-
"created": "2019-08-20T23:02:45.818Z",
113-
"createdBy": "mock-api"
114-
}
115-
info: Created resource: {
116-
"challengeId": "5505f779-b28b-428f-888b-e523b443f3ea",
117-
"memberHandle": "SethHafferkamp",
118-
"roleId": "6605f779-b28b-428f-888b-e523b443f3ea",
119-
"id": "8a835c7e-5a90-47f1-a020-c95c415e32db",
120-
"memberId": "980bd2a6-b4b6-4253-87e7-67cb931a76a2",
121-
"created": "2019-08-20T23:02:45.822Z",
122-
"createdBy": "mock-api"
123-
}
124-
info: Successfully processed message of challenge id 5505f779-b28b-428f-888b-e523b443f3ea and project id 30055214
125+
npm run test
126+
```
127+
To run unit tests with coverage report
128+
```bash
129+
npm run test:cov
130+
```
131+
### Running integration tests
132+
To run integration tests alone
133+
```bash
134+
npm run e2e
135+
```
136+
To run integration tests with coverage report
137+
```bash
138+
npm run e2e:cov
125139
```
126140

127141

128-
- you may write invalid messages like:
129-
`{ "topic": "challenge.notification.create", "originator": "challenge-api", "timestamp": "2019-02-16T00:00:00", "mime-type": "application/json", "payload": { "id": "abc", "typeId": "7705f779-b28b-428f-888b-e523b443f3ea", "track": "code", "name": "test", "description": "desc", "timelineTemplateId": "8805f779-b28b-428f-888b-e523b443f3ea", "phases": [{ "id": "8805f779-b28b-428f-888b-e523b443f3eb", "name": "phase", "isActive": true, "duration": 1000 }], "prizeSets": [{ "type": "Challenge prizes", "prizes": [{ "type": "1st", "value": 600 }] }], "reviewType": "community", "tags": ["tag1"], "projectId": 30055214, "startDate": "2019-02-19T00:00:00", "status": "Draft", "created": "2019-02-16T00:00:00", "createdBy": "tester" } }`
130-
131-
`{ "topic": "challenge.notification.create", "originator": "challenge-api", "timestamp": "2019-02-16T00:00:00", "mime-type": "application/json", "payload": { "id": "5505f779-b28b-428f-888b-e523b443f3ea", "typeId": "7705f779-b28b-428f-888b-e523b443f3ea", "track": "code", "name": "test", "description": "desc", "timelineTemplateId": "8805f779-b28b-428f-888b-e523b443f3ea", "phases": [{ "id": "8805f779-b28b-428f-888b-e523b443f3eb", "name": "phase", "isActive": true, "duration": 1000 }], "prizeSets": [{ "type": "Challenge prizes", "prizes": [{ "type": "1st", "value": 600 }] }], "reviewType": "community", "tags": ["tag1"], "projectId": 30055214, "startDate": "2019-02-19T00:00:00", "status": "Draft", "created": "abc", "createdBy": "tester" } }`
132-
133-
`{ [ { abc`
134-
- then in the app console, you will see error messages
142+
## Running tests in CI
135143

144+
- TBD
136145

137-
- to test the health check API, start the processor, then browse `http://localhost:3000/health` in a browser,
138-
and you will see result `{"checksRun":1}`, you may change the health check API port by setting `PORT` environment variable
146+
## Verification
139147

148+
Refer to the verification document `Verification.md`

‎Verification.md

Lines changed: 47 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,47 @@
1+
2+
## Verification
3+
4+
- setup and start kafka server, start mock API, start processor app
5+
- start kafka-console-producer to write messages to `challenge.notification.create` topic:
6+
`bin/kafka-console-producer.sh --broker-list localhost:9092 --topic challenge.notification.create`
7+
- write message:
8+
`{ "topic": "challenge.notification.create", "originator": "challenge-api", "timestamp": "2019-02-16T00:00:00", "mime-type": "application/json", "payload": { "id": "5505f779-b28b-428f-888b-e523b443f3ea", "typeId": "7705f779-b28b-428f-888b-e523b443f3ea", "track": "code", "name": "test", "description": "desc", "timelineTemplateId": "8805f779-b28b-428f-888b-e523b443f3ea", "phases": [{ "id": "8805f779-b28b-428f-888b-e523b443f3eb", "name": "phase", "isActive": true, "duration": 1000 }], "prizeSets": [{ "type": "Challenge prizes", "prizes": [{ "type": "1st", "value": 600 }] }], "reviewType": "community", "tags": ["tag1"], "projectId": 30055214, "startDate": "2019-02-19T00:00:00", "status": "Draft", "created": "2019-02-16T00:00:00", "createdBy": "tester" } }`
9+
10+
- you will see app logging:
11+
```bash
12+
info: Process message of challenge id 5505f779-b28b-428f-888b-e523b443f3ea and project id 30055214
13+
info: Found member ids [40152933, 40141336] of project id 30055214
14+
info: Created resource: {
15+
"challengeId": "5505f779-b28b-428f-888b-e523b443f3ea",
16+
"memberHandle": "lordofparadox",
17+
"roleId": "6605f779-b28b-428f-888b-e523b443f3ea",
18+
"id": "d4ac5715-b1ed-430b-86ca-56d022c97ce9",
19+
"memberId": "9e2ad182-d4f7-4c4d-9421-29bc8bd56dfb",
20+
"created": "2019-08-20T23:02:45.818Z",
21+
"createdBy": "mock-api"
22+
}
23+
info: Created resource: {
24+
"challengeId": "5505f779-b28b-428f-888b-e523b443f3ea",
25+
"memberHandle": "SethHafferkamp",
26+
"roleId": "6605f779-b28b-428f-888b-e523b443f3ea",
27+
"id": "8a835c7e-5a90-47f1-a020-c95c415e32db",
28+
"memberId": "980bd2a6-b4b6-4253-87e7-67cb931a76a2",
29+
"created": "2019-08-20T23:02:45.822Z",
30+
"createdBy": "mock-api"
31+
}
32+
info: Successfully processed message of challenge id 5505f779-b28b-428f-888b-e523b443f3ea and project id 30055214
33+
```
34+
35+
36+
- you may write invalid messages like:
37+
`{ "topic": "challenge.notification.create", "originator": "challenge-api", "timestamp": "2019-02-16T00:00:00", "mime-type": "application/json", "payload": { "id": "abc", "typeId": "7705f779-b28b-428f-888b-e523b443f3ea", "track": "code", "name": "test", "description": "desc", "timelineTemplateId": "8805f779-b28b-428f-888b-e523b443f3ea", "phases": [{ "id": "8805f779-b28b-428f-888b-e523b443f3eb", "name": "phase", "isActive": true, "duration": 1000 }], "prizeSets": [{ "type": "Challenge prizes", "prizes": [{ "type": "1st", "value": 600 }] }], "reviewType": "community", "tags": ["tag1"], "projectId": 30055214, "startDate": "2019-02-19T00:00:00", "status": "Draft", "created": "2019-02-16T00:00:00", "createdBy": "tester" } }`
38+
39+
`{ "topic": "challenge.notification.create", "originator": "challenge-api", "timestamp": "2019-02-16T00:00:00", "mime-type": "application/json", "payload": { "id": "5505f779-b28b-428f-888b-e523b443f3ea", "typeId": "7705f779-b28b-428f-888b-e523b443f3ea", "track": "code", "name": "test", "description": "desc", "timelineTemplateId": "8805f779-b28b-428f-888b-e523b443f3ea", "phases": [{ "id": "8805f779-b28b-428f-888b-e523b443f3eb", "name": "phase", "isActive": true, "duration": 1000 }], "prizeSets": [{ "type": "Challenge prizes", "prizes": [{ "type": "1st", "value": 600 }] }], "reviewType": "community", "tags": ["tag1"], "projectId": 30055214, "startDate": "2019-02-19T00:00:00", "status": "Draft", "created": "abc", "createdBy": "tester" } }`
40+
41+
`{ [ { abc`
42+
- then in the app console, you will see error messages
43+
44+
45+
- to test the health check API, start the processor, then browse `http://localhost:3000/health` in a browser,
46+
and you will see result `{"checksRun":1}`, you may change the health check API port by setting `PORT` environment variable
47+

‎docs/dev.env

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
DISABLE_LOGGING: '',
3+
LOG_LEVEL: '',
4+
5+
// used to get M2M token
6+
AUTH0_URL: '',
7+
AUTH0_AUDIENCE: '',
8+
TOKEN_CACHE_TIME: '',
9+
AUTH0_PROXY_SERVER_URL: '',
10+
AUTH0_CLIENT_ID: '',
11+
AUTH0_CLIENT_SECRET: '',
12+
13+
KAFKA_URL: '',
14+
KAFKA_GROUP_ID: '',
15+
// below are used for secure Kafka connection, they are optional
16+
// for the local Kafka, they are not needed
17+
KAFKA_CLIENT_CERT: '',
18+
KAFKA_CLIENT_CERT_KEY: '',
19+
KAFKA_TOPIC: '',
20+
21+
// superagent request timeout in milliseconds
22+
REQUEST_TIMEOUT: '',
23+
RESOURCE_ROLE_ID: '',
24+
GET_PROJECT_API_BASE: '',
25+
SEARCH_MEMBERS_API_BASE: '',
26+
CREATE_RESOURCE_API: ''
27+
}

‎docs/prod.env

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
{
2+
DISABLE_LOGGING: '',
3+
LOG_LEVEL: '',
4+
5+
// used to get M2M token
6+
AUTH0_URL: '',
7+
AUTH0_AUDIENCE: '',
8+
TOKEN_CACHE_TIME: '',
9+
AUTH0_PROXY_SERVER_URL: '',
10+
AUTH0_CLIENT_ID: '',
11+
AUTH0_CLIENT_SECRET: '',
12+
13+
KAFKA_URL: '',
14+
KAFKA_GROUP_ID: '',
15+
// below are used for secure Kafka connection, they are optional
16+
// for the local Kafka, they are not needed
17+
KAFKA_CLIENT_CERT: '',
18+
KAFKA_CLIENT_CERT_KEY: '',
19+
KAFKA_TOPIC: '',
20+
21+
// superagent request timeout in milliseconds
22+
REQUEST_TIMEOUT: '',
23+
RESOURCE_ROLE_ID: '',
24+
GET_PROJECT_API_BASE: '',
25+
SEARCH_MEMBERS_API_BASE: '',
26+
CREATE_RESOURCE_API: ''
27+
}

0 commit comments

Comments
 (0)