[Feature Request] Support InternLM Deploy

[vansinhu]

Dear llama.cpp developer,

Greetings! I am vansinhu, a community developer and volunteer at InternLM. InternLM is a large language model similar to llama2, and we look forward to InternLM being supported in llama.cpp. If there are any challenges or inquiries regarding support for InternLM, please feel free to join our Discord discussion at https://discord.gg/gF9ezcmtM3.

Best regards,
vansinhu

[l29ah]

I've tried to run internlm-20b, converted and quantized apparently fine, but when running it i get GGML_ASSERT: llama.cpp:2349: codepoints_from_utf8(word).size() > 0.

[gaord]

try running internLM-chat-20b, convert and quantize fine, but server emit the following error while hosting the model. Any ideas for it working. Thanks.

{"timestamp":1705310958,"level":"INFO","function":"main","line":2971,"message":"HTTP server listening","port":"8086","hostname":"127.0.0.1"}
llama_model_loader: loaded meta data with 19 key-value pairs and 543 tensors from ../internlm-chat-20b-q5_0.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = llama
llama_model_loader: - kv 1: general.name str = LLaMA v2
llama_model_loader: - kv 2: llama.context_length u32 = 4096
llama_model_loader: - kv 3: llama.embedding_length u32 = 5120
llama_model_loader: - kv 4: llama.block_count u32 = 60
llama_model_loader: - kv 5: llama.feed_forward_length u32 = 13824
llama_model_loader: - kv 6: llama.rope.dimension_count u32 = 128
llama_model_loader: - kv 7: llama.attention.head_count u32 = 40
llama_model_loader: - kv 8: llama.attention.head_count_kv u32 = 40
llama_model_loader: - kv 9: llama.attention.layer_norm_rms_epsilon f32 = 0.000001
llama_model_loader: - kv 10: general.file_type u32 = 8
llama_model_loader: - kv 11: tokenizer.ggml.model str = llama
llama_model_loader: - kv 12: tokenizer.ggml.tokens arr[str,103168] = ["", "", "", "<0x00>", "<...
llama_model_loader: - kv 13: tokenizer.ggml.scores arr[f32,103168] = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv 14: tokenizer.ggml.token_type arr[i32,103168] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
llama_model_loader: - kv 15: tokenizer.ggml.bos_token_id u32 = 1
llama_model_loader: - kv 16: tokenizer.ggml.eos_token_id u32 = 2
llama_model_loader: - kv 17: tokenizer.ggml.padding_token_id u32 = 2
llama_model_loader: - kv 18: general.quantization_version u32 = 2
llama_model_loader: - type f32: 121 tensors
llama_model_loader: - type q5_0: 421 tensors
llama_model_loader: - type q6_K: 1 tensors
GGML_ASSERT: llama.cpp:2977: codepoints_from_utf8(word).size() > 0