[SYCL] Fix WARP_SIZE=16 bug of Intel GPU

[luoyu-intel]

• I have read the contributing guidelines
• Self-reported review complexity:
  • Low
  • Medium
  • High

This PR fixes some bugs of WARP_SIZE=16 for Intel GPU. All warp-related UTs are passed.
WARP_SIZE=16 has the same output as WARP_SIZE=32 on Intel GPUs.

NOTE: QX_K kernels are specialized for WARP_SIZE=32, so I use a fixed WARP_SIZE for them.

Performance change

llama-2-7b-chat-hf-q4_0.gguf, 32 in and 32 out, on ARC A770, from 40 tokens/s to 44 tokens/s

Master Branch

 Once upon a time, there existed a little girl, who liked to have adventures. She wanted to go to places and meet new people, have  new experiences and learn new things. She was a curious child, always eager to explore and discover new things.

One day, she found a small door
llama_print_timings:        load time =   14573.13 ms
llama_print_timings:      sample time =       0.53 ms /    32 runs   (    0.02 ms per token, 60952.38 tokens per second)
llama_print_timings: prompt eval time =     217.69 ms /    32 tokens (    6.80 ms per token,   147.00 tokens per second)
llama_print_timings:        eval time =     771.81 ms /    31 runs   (   24.90 ms per token,    40.17 tokens per second)
llama_print_timings:       total time =     994.93 ms /    63 tokens

PR Branch

 Once upon a time, there existed a little girl, who liked to have adventures. She wanted to go to places and meet new people, have  new experiences and learn new things. She was a curious child, always eager to explore and discover new things.

One day, she found a small door
llama_print_timings:        load time =   14716.53 ms
llama_print_timings:      sample time =       0.52 ms /    32 runs   (    0.02 ms per token, 61776.06 tokens per second)
llama_print_timings: prompt eval time =     216.68 ms /    32 tokens (    6.77 ms per token,   147.69 tokens per second)
llama_print_timings:        eval time =     698.20 ms /    31 runs   (   22.52 ms per token,    44.40 tokens per second)
llama_print_timings:       total time =     920.69 ms /    63 tokens

[airMeng]

tested Meta-Llama-3-8B-Instruct-Q4_K_S.gguf and llama-2-7b.Q4_0.gguf

[airMeng]

@joeatodd @OuadiElfarouki

[NeoZhangJianyu]

It's passed on MTL after I test.

[qnixsynapse]

Tested iq4_XS, Q4_K_S.

LGTM

[luoyu-intel]

@qnixsynapse Thanks for your test! Q4_K models still use WARP_SIZE=32, so they won't benefit from this PR.

[qnixsynapse]

@luoyu-intel Yes, I am aware. I am testing IQ4 models currently.

[Alcpz]

@joeatodd @OuadiElfarouki Performance of the SYCL branch using an NVIDIA A100 with Q4_K has no regressions.

model	size	params	backend	ngl	sm	mmap	test	t/s
llama 7B Q4_K - Medium	3.80 GiB	6.74 B	SYCL	78	none	0	pp512	2203.66 ± 15.26
llama 13B Q4_K - Medium	7.33 GiB	13.02 B	SYCL	78	none	0	pp512	1720.49 ± 23.80
llama 70B Q4_K - Medium	38.58 GiB	68.98 B	SYCL	78	none	0	pp512	606.95 ± 5.49

build: 4887fdc (3293)

model	size	params	backend	ngl	sm	mmap	test	t/s
llama 7B Q4_K - Medium	3.80 GiB	6.74 B	SYCL	81	none	0	tg128	5.36 ± 0.00
llama 13B Q4_K - Medium	7.33 GiB	13.02 B	SYCL	81	none	0	tg128	4.27 ± 0.00
llama 70B Q4_K - Medium	38.58 GiB	68.98 B	SYCL	81	none	0	tg128	2.08 ± 0.00

build: 4887fdc (3293)