Add bitnet-embeddings-0.6b model adaptation with F16 and I2_S GGUF conversion

- Add GGUF conversion tool for bitnet-embeddings-0.6b (safetensors -> F16/I2_S GGUF)
- Add Qwen3 architecture support in llama.cpp submodule with per-projection RMSNorm
- Add I2_S ternary quantization (2-bit packed -1/0/+1) for lossless precision
- Add f16 norm weight support for correct embedding inference
- Guard bitnet-lut-kernels.h include with TL1/TL2 preprocessor checks
- Update llama.cpp submodule to dev-bitnet-embedding-0.6b branch
- Document F16 (from multilingual-e5-0.6b) and I2_S (from bitnet-embeddings-0.6b) conversion process
This commit is contained in:
isHuangXin committed 2026-07-12 04:10:30 +02:00
1 parent 01eb415772
commit 69bf64d373
5 files changed
+813 -2

No files matched your search

+7
View File
@@ -5,9 +5,16 @@
#include <stdio.h>
#include <stdlib.h>
#ifdef __x86_64__
#include <immintrin.h>
#endif
#include "ggml-bitnet.h"
#include "ggml-quants.h"
#if defined(GGML_BITNET_ARM_TL1) || defined(GGML_BITNET_X86_TL2)
#include "bitnet-lut-kernels.h"
#endif
#if defined(GGML_BITNET_ARM_TL1)
+1 -1
View File
@@ -808,7 +808,7 @@ void ggml_vec_dot_i2_i8_s_Nx1(int n, float * s, size_t bs, const void * vx, size
accu[iy] = _mm256_setzero_si256();
}
int8_t * y_col = y + col * by;
const int8_t * y_col = y + col * by;
for (int i = 0; i < group32_num; i++) {
const uint8_t *px = x + i * 1024;