Description
I started scrapyd , mongo and rabbitmq in worker threads still it shows worker status: unreachable , version: unknown
When I cal ping workers from Link generator curl working file
This is how ./scrapyproject/scrapy_packages/settings.py looks
`SCHEDULER = ".rabbitmq.scheduler.Scheduler"
SCHEDULER_PERSIST = True
RABBITMQ_HOST = '127.0.0.1'
RABBITMQ_PORT = 5672
RABBITMQ_USERNAME = 'guest'
RABBITMQ_PASSWORD = 'guest'
MONGODB_PUBLIC_ADDRESS = '127.0.0.1:27017' # This will be shown on the web intee
rface, but won't be used for connecting to DB
MONGODB_URI = '127.0.0.1:27017' # Actual uri to connect to DB
MONGODB_USER = ''
MONGODB_PASSWORD = ''
MONGODB_SHARDED = False
MONGODB_BUFFER_DATA = 100
LINK_GENERATOR = 'http://127.0.0.1:6800' # Set your link generator worker addree
ss here
SCRAPERS = ['172.17.0.4:3800', '172.17.0.2:6800'] # Set your scraper worker addd
resses here
LINUX_USER_CREATION_ENABLED = False # Set this to True if you want a linux userr
account created during registration`
And This is what I get when I curl them from link generator -
`root@96f3b9b83573:/usr/local/Distributed-Multi-User-Scrapy-System-with-a-Web-UI#
curl -I 172.17.0.4:3800
HTTP/1.1 200 OK
Date: Fri, 11 May 2018 04:50:31 GMT
Content-Length: 699
Content-Type: text/html
Server: TwistedWeb/18.4.0
root@96f3b9b83573:/usr/local/Distributed-Multi-User-Scrapy-System-with-a-Web-UI#
curl -I 172.17.0.2:6800
HTTP/1.1 200 OK
Date: Fri, 11 May 2018 04:50:50 GMT
Content-Length: 699
Content-Type: text/html
Server: TwistedWeb/18.4.0`
So workers are up .. but somehow link generator cant connect to them.