Skip to content

Move dataset generation before switching to minikube docker daemon. #2144

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 17 additions & 18 deletions docs/tutorials/elasticdl_local.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,15 +34,30 @@ Clone elasticdl repo for model zoo and some scripts.
git clone https://github.com/sql-machine-learning/elasticdl.git
```

### Prepare the dataset

We generate MNIST training and evaluation data in RecordIO format. We provide a
script in elasticdl repo.

```bash
docker pull elasticdl/elasticdl:dev
# Change directory to the root of elasticdl repo
cd elasticdl
mkdir data
docker run --rm -it \
-v $HOME/.keras/datasets:/root/.keras/datasets \
-v $PWD:/work \
-w /work elasticdl/elasticdl:dev \
bash -c "scripts/gen_dataset.sh data"
```

### Start Kubernetes Cluster

We start minikube with a command-line option `--mount-string`, which mounts the
directory `{elasticdl_repo_root}/data` in local host to `/data` path in all
minikube containers.

```bash
cd elasticdl
mkdir data
minikube start --vm-driver=hyperkit --cpus 2 --memory 6144 --disk-size=50gb --mount=true --mount-string="./data:/data"
kubectl apply -f elasticdl/manifests/elasticdl-rbac.yaml
eval $(minikube docker-env)
Expand All @@ -59,22 +74,6 @@ elasticdl zoo build --image=elasticdl:mnist .
We use the model predefined in model zoo directory. The model definition will
be packed into the new Docker image `elasticdl:mnist`.

### Prepare the dataset

We generate MNIST training and evaluation data in RecordIO format. We provide a
script in elasticdl repo.

```bash
# Change directory to the root of elasticdl repo
cd ../
docker pull elasticdl/elasticdl:dev
docker run --rm -it \
-v $HOME/.keras/datasets:/root/.keras/datasets \
-v $PWD:/work \
-w /work elasticdl/elasticdl:dev \
bash -c "scripts/gen_dataset.sh data"
```

### Summit a training job

We use the following command to submit a training job:
Expand Down