Skip to content

work period endpoint #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Apr 4, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,9 @@ The following parameters can be set in config files or in env variables:
- `topics.TAAS_RESOURCE_BOOKING_CREATE_TOPIC`: the create resource booking entity Kafka message topic
- `topics.TAAS_RESOURCE_BOOKING_UPDATE_TOPIC`: the update resource booking entity Kafka message topic
- `topics.TAAS_RESOURCE_BOOKING_DELETE_TOPIC`: the delete resource booking entity Kafka message topic
- `topics.TAAS_WORK_PERIOD_CREATE_TOPIC`: the create work period entity Kafka message topic
- `topics.TAAS_WORK_PERIOD_UPDATE_TOPIC`: the update work period entity Kafka message topic
- `topics.TAAS_WORK_PERIOD_DELETE_TOPIC`: the delete work period entity Kafka message topic
- `esConfig.HOST`: Elasticsearch host
- `esConfig.AWS_REGION`: The Amazon region to use when using AWS Elasticsearch service
- `esConfig.ELASTICCLOUD.id`: The elastic cloud id, if your elasticsearch instance is hosted on elastic cloud. DO NOT provide a value for ES_HOST if you are using this
Expand All @@ -38,6 +41,7 @@ The following parameters can be set in config files or in env variables:
- `esConfig.ES_INDEX_JOB`: the index name for job
- `esConfig.ES_INDEX_JOB_CANDIDATE`: the index name for job candidate
- `esConfig.ES_INDEX_RESOURCE_BOOKING`: the index name for resource booking
- `esConfig.ES_INDEX_WORK_PERIOD`: the index name for work period

- `auth0.AUTH0_URL`: Auth0 URL, used to get TC M2M token
- `auth0.AUTH0_AUDIENCE`: Auth0 audience, used to get TC M2M token
Expand Down
12 changes: 9 additions & 3 deletions VERIFICATION.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

## Create documents in ES

- Run the following commands to create `Job`, `JobCandidate` and `ResourceBooking` documents in ES.
- Run the following commands to create `Job`, `JobCandidate`, `ResourceBooking`, `WorkPeriod` documents in ES.

``` bash
# for Job
Expand All @@ -11,12 +11,14 @@
docker exec -i taas-es-processor_kafka /opt/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic taas.jobcandidate.create < test/messages/taas.jobcandidate.create.event.json
# for ResourceBooking
docker exec -i taas-es-processor_kafka /opt/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic taas.resourcebooking.create < test/messages/taas.resourcebooking.create.event.json
# for WorkPeriod
docker exec -i taas-es-processor_kafka /opt/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic taas.workperiod.create < test/messages/taas.workperiod.create.event.json
```

- Run `npm run view-data <model-name-here>` to see if documents were created.

## Update documents in ES
- Run the following commands to update `Job`, `JobCandidate` and `ResourceBooking` documents in ES.
- Run the following commands to update `Job`, `JobCandidate`, `ResourceBooking`, `WorkPeriod` documents in ES.

``` bash
# for Job
Expand All @@ -25,12 +27,14 @@
docker exec -i taas-es-processor_kafka /opt/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic taas.jobcandidate.update < test/messages/taas.jobcandidate.update.event.json
# for ResourceBooking
docker exec -i taas-es-processor_kafka /opt/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic taas.resourcebooking.update < test/messages/taas.resourcebooking.update.event.json
# for WorkPeriod
docker exec -i taas-es-processor_kafka /opt/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic taas.workperiod.update < test/messages/taas.workperiod.update.event.json
```

- Run `npm run view-data <model-name-here>` to see if documents were updated.

## Delete documents in ES
- Run the following commands to delete `Job`, `JobCandidate` and `ResourceBooking` documents in ES.
- Run the following commands to delete `Job`, `JobCandidate`, `ResourceBooking`, `WorkPeriod` documents in ES.

``` bash
# for Job
Expand All @@ -39,6 +43,8 @@
docker exec -i taas-es-processor_kafka /opt/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic taas.jobcandidate.delete < test/messages/taas.jobcandidate.delete.event.json
# for ResourceBooking
docker exec -i taas-es-processor_kafka /opt/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic taas.resourcebooking.delete < test/messages/taas.resourcebooking.delete.event.json
# for WorkPeriod
docker exec -i taas-es-processor_kafka /opt/kafka/bin/kafka-console-producer.sh --broker-list localhost:9092 --topic taas.workperiod.delete < test/messages/taas.workperiod.delete.event.json
```

- Run `npm run view-data <model-name-here>` to see if documents were deleted.
12 changes: 9 additions & 3 deletions config/default.js
Original file line number Diff line number Diff line change
Expand Up @@ -16,17 +16,22 @@ module.exports = {
KAFKA_GROUP_ID: process.env.KAFKA_GROUP_ID || 'taas-es-processor',

topics: {
// topics for job service
TAAS_JOB_CREATE_TOPIC: process.env.TAAS_JOB_CREATE_TOPIC || 'taas.job.create',
TAAS_JOB_UPDATE_TOPIC: process.env.TAAS_JOB_UPDATE_TOPIC || 'taas.job.update',
TAAS_JOB_DELETE_TOPIC: process.env.TAAS_JOB_DELETE_TOPIC || 'taas.job.delete',
// topics for jobcandidate service
TAAS_JOB_CANDIDATE_CREATE_TOPIC: process.env.TAAS_JOB_CANDIDATE_CREATE_TOPIC || 'taas.jobcandidate.create',
TAAS_JOB_CANDIDATE_UPDATE_TOPIC: process.env.TAAS_JOB_CANDIDATE_UPDATE_TOPIC || 'taas.jobcandidate.update',
TAAS_JOB_CANDIDATE_DELETE_TOPIC: process.env.TAAS_JOB_CANDIDATE_DELETE_TOPIC || 'taas.jobcandidate.delete',
// topics for job service
// topics for resource booking service
TAAS_RESOURCE_BOOKING_CREATE_TOPIC: process.env.TAAS_RESOURCE_BOOKING_CREATE_TOPIC || 'taas.resourcebooking.create',
TAAS_RESOURCE_BOOKING_UPDATE_TOPIC: process.env.TAAS_RESOURCE_BOOKING_UPDATE_TOPIC || 'taas.resourcebooking.update',
TAAS_RESOURCE_BOOKING_DELETE_TOPIC: process.env.TAAS_RESOURCE_BOOKING_DELETE_TOPIC || 'taas.resourcebooking.delete'
TAAS_RESOURCE_BOOKING_DELETE_TOPIC: process.env.TAAS_RESOURCE_BOOKING_DELETE_TOPIC || 'taas.resourcebooking.delete',
// topics for work period service
TAAS_WORK_PERIOD_CREATE_TOPIC: process.env.TAAS_WORK_PERIOD_CREATE_TOPIC || 'taas.workperiod.create',
TAAS_WORK_PERIOD_UPDATE_TOPIC: process.env.TAAS_WORK_PERIOD_UPDATE_TOPIC || 'taas.workperiod.update',
TAAS_WORK_PERIOD_DELETE_TOPIC: process.env.TAAS_WORK_PERIOD_DELETE_TOPIC || 'taas.workperiod.delete'
},

esConfig: {
Expand All @@ -42,7 +47,8 @@ module.exports = {

ES_INDEX_JOB: process.env.ES_INDEX_JOB || 'job',
ES_INDEX_JOB_CANDIDATE: process.env.ES_INDEX_JOB_CANDIDATE || 'job_candidate',
ES_INDEX_RESOURCE_BOOKING: process.env.ES_INDEX_RESOURCE_BOOKING || 'resource_booking'
ES_INDEX_RESOURCE_BOOKING: process.env.ES_INDEX_RESOURCE_BOOKING || 'resource_booking',
ES_INDEX_WORK_PERIOD: process.env.ES_INDEX_WORK_PERIOD || 'work_period'
},

auth0: {
Expand Down
7 changes: 6 additions & 1 deletion src/app.js
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ const helper = require('./common/helper')
const JobProcessorService = require('./services/JobProcessorService')
const JobCandidateProcessorService = require('./services/JobCandidateProcessorService')
const ResourceBookingProcessorService = require('./services/ResourceBookingProcessorService')
const WorkPeriodProcessorService = require('./services/WorkPeriodProcessorService')
const Mutex = require('async-mutex').Mutex
const events = require('events')

Expand All @@ -38,7 +39,11 @@ const topicServiceMapping = {
// resource booking
[config.topics.TAAS_RESOURCE_BOOKING_CREATE_TOPIC]: ResourceBookingProcessorService.processCreate,
[config.topics.TAAS_RESOURCE_BOOKING_UPDATE_TOPIC]: ResourceBookingProcessorService.processUpdate,
[config.topics.TAAS_RESOURCE_BOOKING_DELETE_TOPIC]: ResourceBookingProcessorService.processDelete
[config.topics.TAAS_RESOURCE_BOOKING_DELETE_TOPIC]: ResourceBookingProcessorService.processDelete,
// work period
[config.topics.TAAS_WORK_PERIOD_CREATE_TOPIC]: WorkPeriodProcessorService.processCreate,
[config.topics.TAAS_WORK_PERIOD_UPDATE_TOPIC]: WorkPeriodProcessorService.processUpdate,
[config.topics.TAAS_WORK_PERIOD_DELETE_TOPIC]: WorkPeriodProcessorService.processDelete
}

// Start kafka consumer
Expand Down
1 change: 1 addition & 0 deletions src/bootstrap.js
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,7 @@ Joi.jobStatus = () => Joi.string().valid('sourcing', 'in-review', 'assigned', 'c
Joi.jobCandidateStatus = () => Joi.string().valid('open', 'selected', 'shortlist', 'rejected', 'cancelled', 'interview')
Joi.workload = () => Joi.string().valid('full-time', 'fractional')
Joi.title = () => Joi.string().max(128)
Joi.paymentStatus = () => Joi.string().valid('pending', 'partially-completed', 'completed', 'cancelled')
// Empty string is not allowed by Joi by default and must be enabled with allow('').
// See https://joi.dev/api/?v=17.3.0#string fro details why it's like this.
// In many cases we would like to allow empty string to make it easier to create UI for editing data.
Expand Down
23 changes: 23 additions & 0 deletions src/scripts/createIndex.js
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,29 @@ async function createIndex () {
}
}
}
},
{
index: config.get('esConfig.ES_INDEX_WORK_PERIOD'),
body: {
mappings: {
properties: {
resourceBookingId: { type: 'keyword' },
userHandle: { type: 'keyword' },
projectId: { type: 'integer' },
userId: { type: 'keyword' },
startDate: { type: 'date', format: 'yyyy-MM-dd' },
endDate: { type: 'date', format: 'yyyy-MM-dd' },
daysWorked: { type: 'integer' },
memberRate: { type: 'float' },
customerRate: { type: 'float' },
paymentStatus: { type: 'keyword' },
createdAt: { type: 'date' },
createdBy: { type: 'keyword' },
updatedAt: { type: 'date' },
updatedBy: { type: 'keyword' }
}
}
}
}]

for (const index of indices) {
Expand Down
3 changes: 2 additions & 1 deletion src/scripts/deleteIndex.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ async function deleteIndex () {
const esClient = helper.getESClient()
const indices = [config.get('esConfig.ES_INDEX_JOB'),
config.get('esConfig.ES_INDEX_JOB_CANDIDATE'),
config.get('esConfig.ES_INDEX_RESOURCE_BOOKING')]
config.get('esConfig.ES_INDEX_RESOURCE_BOOKING'),
config.get('esConfig.ES_INDEX_WORK_PERIOD')]
for (const index of indices) {
await esClient.indices.delete({
index
Expand Down
3 changes: 2 additions & 1 deletion src/scripts/view-data.js
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,8 @@ const esClient = helper.getESClient()
const modelIndexMapping = {
Job: 'ES_INDEX_JOB',
JobCandidate: 'ES_INDEX_JOB_CANDIDATE',
ResourceBooking: 'ES_INDEX_RESOURCE_BOOKING'
ResourceBooking: 'ES_INDEX_RESOURCE_BOOKING',
WorkPeriod: 'ES_INDEX_WORK_PERIOD'
}

async function showESData () {
Expand Down
109 changes: 109 additions & 0 deletions src/services/WorkPeriodProcessorService.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,109 @@
/**
* WorkPeriod Processor Service
*/

const Joi = require('@hapi/joi')
const logger = require('../common/logger')
const helper = require('../common/helper')
const constants = require('../common/constants')
const config = require('config')

const esClient = helper.getESClient()

/**
* Process create entity message
* @param {Object} message the kafka message
* @param {String} transactionId
*/
async function processCreate (message, transactionId) {
const workPeriod = message.payload
await esClient.createExtra({
index: config.get('esConfig.ES_INDEX_WORK_PERIOD'),
id: workPeriod.id,
transactionId,
body: workPeriod,
refresh: constants.esRefreshOption
})
}

processCreate.schema = {
message: Joi.object().keys({
topic: Joi.string().required(),
originator: Joi.string().required(),
timestamp: Joi.date().required(),
'mime-type': Joi.string().required(),
payload: Joi.object().keys({
id: Joi.string().uuid().required(),
resourceBookingId: Joi.string().uuid().required(),
userHandle: Joi.string().required(),
projectId: Joi.number().integer().required(),
startDate: Joi.string().required(),
endDate: Joi.string().required(),
daysWorked: Joi.number().integer().min(0).allow(null),
memberRate: Joi.number().allow(null),
customerRate: Joi.number().allow(null),
paymentStatus: Joi.paymentStatus().required(),
createdAt: Joi.date().required(),
createdBy: Joi.string().uuid().required(),
updatedAt: Joi.date().allow(null),
updatedBy: Joi.string().uuid().allow(null)
}).required()
}).required(),
transactionId: Joi.string().required()
}

/**
* Process update entity message
* @param {Object} message the kafka message
* @param {String} transactionId
*/
async function processUpdate (message, transactionId) {
const data = message.payload
await esClient.updateExtra({
index: config.get('esConfig.ES_INDEX_WORK_PERIOD'),
id: data.id,
transactionId,
body: {
doc: data
},
refresh: constants.esRefreshOption
})
}

processUpdate.schema = processCreate.schema

/**
* Process delete entity message
* @param {Object} message the kafka message
* @param {String} transactionId
*/
async function processDelete (message, transactionId) {
const id = message.payload.id
await esClient.deleteExtra({
index: config.get('esConfig.ES_INDEX_WORK_PERIOD'),
id,
transactionId,
refresh: constants.esRefreshOption
})
}

processDelete.schema = {
message: Joi.object().keys({
topic: Joi.string().required(),
originator: Joi.string().required(),
timestamp: Joi.date().required(),
'mime-type': Joi.string().required(),
payload: Joi.object().keys({
id: Joi.string().uuid().required()
}).required()
}).required(),
transactionId: Joi.string().required()
}

module.exports = {
processCreate,
processUpdate,
processDelete
}

logger.buildService(module.exports, 'WorkPeriodProcessorService')
22 changes: 22 additions & 0 deletions test/messages/taas.workperiod.create.event.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
{
"topic": "taas.workperiod.create",
"originator": "taas-api",
"timestamp": "2021-03-30T20:24:17.555Z",
"mime-type": "application/json",
"payload": {
"resourceBookingId": "6cf2edf6-4b2c-40ef-96db-e1ddb771fdd3",
"startDate": "2021-03-14",
"endDate": "2021-03-20",
"daysWorked": 3,
"memberRate": 13.13,
"customerRate": 13.13,
"paymentStatus": "cancelled",
"projectId": 111,
"userHandle": "pshah_manager",
"id": "926040c4-1709-4de2-b2b6-52adf6e5e72d",
"createdBy": "00000000-0000-0000-0000-000000000000",
"updatedAt": "2021-03-30T20:24:17.541Z",
"createdAt": "2021-03-30T20:24:17.541Z",
"updatedBy": null
}
}
9 changes: 9 additions & 0 deletions test/messages/taas.workperiod.delete.event.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
{
"topic": "taas.workperiod.delete",
"originator": "taas-api",
"timestamp": "2021-03-30T20:13:58.491Z",
"mime-type": "application/json",
"payload": {
"id": "926040c4-1709-4de2-b2b6-52adf6e5e72d"
}
}
21 changes: 21 additions & 0 deletions test/messages/taas.workperiod.update.event.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
{
"topic": "taas.workperiod.update",
"originator": "taas-api",
"timestamp": "2021-03-30T20:13:53.179Z",
"mime-type": "application/json",
"payload": {
"id": "926040c4-1709-4de2-b2b6-52adf6e5e72d",
"resourceBookingId": "79317ff6-5b30-45c2-ace8-b97282b042a8",
"startDate": "2021-03-14",
"endDate": "2021-03-20",
"daysWorked": 3,
"memberRate": 13.13,
"customerRate": 13.13,
"paymentStatus": "pending",
"projectId": 111,
"userHandle": "pshah_manager",
"createdBy": "00000000-0000-0000-0000-000000000000",
"createdAt": "2021-03-30T20:13:34.670Z",
"updatedAt": "2021-03-30T20:13:45.354Z"
}
}