PyGAD on GPU

[IMG-5]

Salam Ahmed. Can I implement the PyGAD framework on Nvidia GPU instead of CPU using RAPIDA, numba and CUDA? If not, do you know of a genetic algorithm library in Python that can be implemented on GPU? Thank you.

[BenoitMiquey]

Hi @IMG-5 ,
I saw in torchga.py for exemple, the usage of cpu is forced, but i think that's possible to edit the sources to use GPU.
But i'm not sure that using a gpu instead of cpu is a good way in this case.

[whubaichuan]

PyGad for genetic algorithm (pygad Module) currently does not support GPU I guess.

[ahmedfgad]

@IMG-5 I think you mean solving the optimization problem itself using a GPU (i.e. using GPU with the pygad module itself). This is not currently supported. You are right @whubaichuan.

As @BenoitMiquey mentioned, using the GPU might not be feasible. The operations in the genetic algorithm, like crossover and mutation, is not computationally expensive.

But the GPU support is indeed a good feature to the pygad.kerasga and pygad.torchga.

[whubaichuan]

@IMG-5 it also depends on which fitness_function you have. Unless it's a big DL network to be trained with lots of data, most of the time, the CPU may be quicker than GPU and you can use built-in multiprocess to speed up.

[IMG-5]

Thank you, @whubaichuan and @ahmedfgad. I managed to build my function, which I am optimizing, using the GPU libraries (i.e., cudf and cupy) and Pygad works fine with that. However, I am referring here to the actual Pygad process (mutation and crossover), which is based on numby. If the Pygad prerequisite libraries are replaced with the GPU counterparts (i.e., replacing pandas with cudf and numby with cupy), will that make the mutation and crossover faster? I read that parallel computing (GPUs) does now work properly if the solution i depends on i-1, which I think is the case in Pygad's crossover and mutation.
The process of mutation and crossover is actually slow when I use these settings, this is because I have 70 genes:
num_parents_mating=60
sol_per_pop=120
However, solving my model is fast, so I am guessing that Pygad is the bottleneck in this case. If I reduce the number of parents and sol per pop, I get faster iterations but then I would get stuck in a local optima and I will have to manually change the ranges of my 70 genes. I though if Pygad could run on GPUs then we optimize large problems efficiently but I might be wrong.
If GPU is not possible, can you recommend CPU specifications that would accelerate Pygad mutation and crossover?
P.S. I found that both 'process' and 'thread' settings in parallel_processing would not reduce the computation time of my problem. I am not sure if there are other settings that I should be experimenting with.

[ahmedfgad]

@IMG-5, according to the experiments we made, we found that crossover and mutation do not take much processing time. The bottleneck is the fitness function.

At the moment, parallel processing is supported only to calculate the fitness. But it is not applied to crossover and mutation as this increases the overall time.

[whubaichuan]

@IMG-5 you set sol_per_pop=120, it needs 120 times to calculate the fitness. if you also set parallel_processing=['process', 10] (if you have a 10-core CPU), then it needs around 12 times to calculate the fitness theoretically.

By the way, if you need GPU to calculate fitness, you can consider NNI library for Microsoft.