|
1 | 1 | # Manual Ingestion
|
2 | 2 |
|
3 |
| -Given below are the pre-requisites for MySQL virtualization when using Manual Ingestion mode. |
4 |
| - |
5 |
| -### Source Environment Requirements |
6 |
| - |
7 |
| -Source environment is where the source MySQL databases are running. |
8 |
| - |
9 |
| -#### Source DB User |
10 |
| -In Manual Ingestion mode, Delphix does not interact with the source database. |
11 |
| -Delphix creates a staging database and the backups are ingested manually by a customer user. |
12 |
| - |
13 |
| -However, Delphix needs to be able to manage this staging database for snapshots and other time travel operations. |
14 |
| -Hence, we require a user with the following permissions on the staging database. |
15 |
| - |
16 |
| - *SELECT, SHUTDOWN, SUPER, RELOAD ,SHOW VIEW, EVENT, TRIGGER* |
17 |
| - |
18 |
| -There are 2 ways to achieve this |
19 |
| - |
20 |
| -1. Create this user on your source database so that when the backup is restored, this user is present in the staging db. |
21 |
| -2. Create this user manually in the staging db. |
22 |
| - In this case, the customer user must ensure that this user is always present in the staging db |
23 |
| - and has the necessary privileges |
24 |
| - |
25 |
| - ```jql |
26 |
| - mysql> GRANT SELECT, SHUTDOWN, SUPER, RELOAD ,SHOW VIEW, EVENT, TRIGGER on *.* to 'user'@'staging-host'; |
27 |
| - ``` |
28 |
| - You can also grant more permissive privileges |
29 |
| - |
30 |
| - ```jql |
31 |
| - mysql>GRANT ALL PRIVILEGES ON *.* TO 'user'@'%'; |
32 |
| - ``` |
33 |
| - |
34 |
| -!!! note |
35 |
| - Remember, this is the user that Delphix uses to manage the Staging database. |
36 |
| - So, if you're using Option#1 above, it is recommended that you create a dedicated source db user for Delphix with the privileges |
37 |
| - mentioned above. |
38 |
| - |
39 |
| -### Staging Environment Requirements |
40 |
| - |
41 |
| -#### Staging OS User |
42 |
| -- A Delphix OS user with the elevated permissions to run *ps, mount, umount, mkdir, rmdir* |
43 |
| - commands without requiring a password |
44 |
| - |
45 |
| - <div class="code_box_outer"> |
46 |
| - <div class="code_box_title"> |
47 |
| - <span class="code_title">*/etc/sudoers*</span> |
48 |
| - </div> |
49 |
| - <div> |
50 |
| - ```groovy hl_lines="6" |
51 |
| - Defaults:delphix_os !requiretty |
52 |
| - delphix_os ALL=NOPASSWD: \ |
53 |
| - /bin/mount, /bin/umount, /bin/mkdir, /bin/rmdir, /bin/ps |
54 |
| - ``` |
55 |
| - </div> |
56 |
| - </div> |
57 |
| - |
58 |
| -- Delphix OS user should be in the same primary and secondary groups as mysql user ( or the MySQL binary owner ) |
59 |
| -- Delphix OS user must have execute access on all files within MySQL installation folder - Min permission level 775 recommended. |
60 |
| - |
61 |
| -#### Storage |
62 |
| -- Empty folder on staging host to hold delphix toolkit [ approximate 2GB free space ] |
63 |
| - |
64 |
| -#### MySQL Version & Configuration |
65 |
| -- MySQL Binary version must match the version on the source database(s) |
66 |
| - |
67 |
| -- Starter *my.cnf* file <span class="code_title">[Recommended] </span> |
68 |
| - |
69 |
| - As every organization's MySQL configuration is different, |
70 |
| - Delphix expects a starter *my.cnf* file to be present in Delphix Toolkit Directory when creating a staging database. |
71 |
| - |
72 |
| - Delphix will use this *my.cnf* file and modify it as per the configuration provided during the the dsource creation process. |
73 |
| - |
74 |
| - This is recommended to reduce the possibility of errors while restoring the backup from the source database. |
75 |
| - |
76 |
| - if tnis file is not provided, Delphix will create a *my.cnf* file from scratch. |
77 |
| - |
78 |
| -### Target Environment Requirements |
79 |
| - |
80 |
| -#### Target OS User |
81 |
| -- A Delphix OS user with the elevated permissions to run *ps, mount, umount, mkdir, rmdir* |
82 |
| - commands without requiring a password |
83 |
| - |
84 |
| - <div class="code_box_outer"> |
85 |
| - <div class="code_box_title"> |
86 |
| - <span class="code_title">*/etc/sudoers*</span> |
87 |
| - </div> |
88 |
| - <div> |
89 |
| - ```groovy hl_lines="6" |
90 |
| - Defaults:delphix_os !requiretty |
91 |
| - delphix_os ALL=NOPASSWD: \ |
92 |
| - /bin/mount, /bin/umount, /bin/mkdir, /bin/rmdir, /bin/ps |
93 |
| - ``` |
94 |
| - </div> |
95 |
| - </div> |
96 |
| - |
97 |
| -- Delphix OS user should be in the same primary and secondary groups as mysql user ( or the MySQL binary owner ) |
98 |
| -- Delphix OS user must have execute access on all files within MySQL installation folder - Min permission level 775 recommended. |
| 3 | +In Manual Ingestion Mode, Delphix creates a seed (empty) staging database. |
| 4 | +Customer users will be responsible to manually ingest data into this staging db. |
| 5 | +All other management operations can be performed through Delphix. A snapshot can be taken either manually |
| 6 | +or via a SnapSync policy once the data is restored into the seed staging database. |
99 | 7 |
|
| 8 | +### Pre-Requisites |
| 9 | + |
| 10 | +- Staging environment must be added to Delphix. |
| 11 | +- A Source Config must be created on the staging environment-MySQL repository. |
| 12 | +- <span class="code_title">[Recommended] </span> |
| 13 | + As every organization's MySQL configuration is different, |
| 14 | + Delphix expects a starter *my.cnf* file to be present in Delphix Toolkit Directory when creating a staging database. |
| 15 | + Delphix will use this *my.cnf* file and modify it as per the configuration provided during the the dsource creation process. |
| 16 | + |
| 17 | + This is recommended to reduce the possibility of errors while restoring the backup from the source database. |
| 18 | + |
| 19 | + !!! warning |
| 20 | + It may take upto 5 minutes after successful dSource creation for the status to show as Active. |
| 21 | + |
| 22 | + |
| 23 | +### Creating dSource |
| 24 | + |
| 25 | +1. Login to **Delphix Management** application. |
| 26 | +2. Click **Manage** > **Datasets**. |
| 27 | +3. Select **Add dSource**. |
| 28 | + |
| 29 | +  |
| 30 | + |
| 31 | + |
| 32 | +4. In the Add dSource wizard, select the MySQL source configuration which is created on the staging host. |
| 33 | +5. Select *Manual Ingestion* in the dSource Type dropdown. |
| 34 | +6. Provide the additional details required for dsource creation |
| 35 | + - Staging DB Server ID |
| 36 | + |
| 37 | + Server ID for the dsource (stagind db. For Replication Mode, |
| 38 | + this server id must be greater than the source db server id. |
| 39 | + |
| 40 | + - Staging DB Port |
| 41 | + |
| 42 | + Port for the dsource (staging db). |
| 43 | + |
| 44 | + - Staging Initialization Password |
| 45 | + |
| 46 | + Password to use while initializing the dsource (staging db). |
| 47 | + This password will be assigned to the *'root'@'localhost'* |
| 48 | + |
| 49 | + - MySQL Base Directory |
| 50 | + |
| 51 | + MySQL installation directory. This is where the location of */bin/mysql* |
| 52 | + |
| 53 | + - Mount Location on Staging Host |
| 54 | + |
| 55 | + This is the mount directory for Delphix on the staging host. |
| 56 | + This location should be unique and empty. |
| 57 | + |
| 58 | + - Source DB UserName |
| 59 | + |
| 60 | + Delphix db user on the source database. In Manual Ingestion mode, Delphix does not connect to the source db. |
| 61 | + Delphix will be using this db user to manage the dsource and other time travel operations. |
| 62 | + This user must be part of the source db backup that will be restored into the staging db. |
| 63 | + |
| 64 | + - Source DB Password |
| 65 | + |
| 66 | + Password for the source db user. |
| 67 | + |
| 68 | +8. On dSource Configuration screen, select the dataset group where the dSource will be placed and click *Next*. |
| 69 | +9. On the Data Management screen, select the staging environment and the environment user and click *Next*. |
| 70 | +10. On the Policies screen, select the Snapsync and Retention policies for the dSource and click *Next*. |
| 71 | +11. On the Hooks screen, add any pre-sync and (or) post-sync hooks as required and click *Next*. |
| 72 | +12. Review the dSource configuration on the Summary screen and Submit. The Linking process has commenced. |
| 73 | + |
| 74 | +  |
| 75 | + |
| 76 | + |
| 77 | +Once the dSource creation is successful, the <span class="code_title">*Timeflow*</span> tab should show the initial snapshot. |
100 | 78 |
|
101 | 79 | Done, What's Next?
|
102 | 80 | ----------------
|
103 |
| -Alright, we've taken care of all the pre-requisites. Next step - Install the plugin. |
| 81 | +Alright, we have created a MySQL dSource. Next step - Provision a VDB. |
104 | 82 |
|
0 commit comments