lee/BitNet
1
0
Fork 0
mirror of https://github.com/microsoft/BitNet.git synced 2026-08-23 17:36:31 +00:00
Code Issues Packages Projects Releases Wiki Activity
103 Commits 7 Branches 0 Tags
85a507c67dd2e361f94152c1e8babff2cb4e7daf
Commit Graph
103 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
isHuangXin 85a507c67d Fix vocab encoding by disabling add_special_tokens in tokenizer.encode 2026-07-16 05:32:29 +02:00
isHuangXin 403efc7b00 Fix inference pipeline: remove hardcoded batch size, enable tool builds, update Python requirement 2026-07-15 11:59:58 +02:00
isHuangXin 425fbbcb75 Update submodule to release-bitnet-embedding-0.6b-270m branch 2026-07-15 10:30:33 +02:00
isHuangXin 1f6a25aeeb Update llama.cpp submodule to include SafetensorRemote utility classes 2026-07-15 08:47:54 +02:00
isHuangXin ce530b0fd0 Add bitnet-embeddings-270m model adaptation with F16 and I2_S GGUF conversion 
- Add LLM_ARCH_GEMMA3 in llama.cpp for gemma3_text model type
  (embedding scaling, GELU, post-attn/post-FFN norms, GQA)
- Add GGUF conversion support for Gemma3-based 270m models
  (SPM tokenizer, RMSNorm w+1 offset, arch-specific tensor mapping)
- Add tokenizer hash for multilingual-e5-0.6b-260311
- Add conversion documentation
 2026-07-15 08:29:16 +02:00
isHuangXin 3b04140a54 feat: add I2_S GGUF conversion for bitnet-b1.58-2B-4T and refactor T-MAC LUT path 
- Add quantize_to_i2_s() for direct ternary-to-I2_S packing in conversion script
- Support offline-quantized models (uint8 packed weights + weight_scale)
- Fix weight_quant double-quantization bug for offline-quantized models
- Fix I2_S scale computation to use first nonzero absolute value
- Add I2_S ftype mapping and BitNetForCausalLM registration
- Refactor ggml-bitnet-lut T-MAC wrapper with proper mul_mat implementation
- Update llama.cpp submodule with I2_S ftype and 2B model type support
 2026-07-13 06:26:16 +02:00
isHuangXin 69bf64d373 Add bitnet-embeddings-0.6b model adaptation with F16 and I2_S GGUF conversion 
- Add GGUF conversion tool for bitnet-embeddings-0.6b (safetensors -> F16/I2_S GGUF)
- Add Qwen3 architecture support in llama.cpp submodule with per-projection RMSNorm
- Add I2_S ternary quantization (2-bit packed -1/0/+1) for lossless precision
- Add f16 norm weight support for correct embedding inference
- Guard bitnet-lut-kernels.h include with TL1/TL2 preprocessor checks
- Update llama.cpp submodule to dev-bitnet-embedding-0.6b branch
- Document F16 (from multilingual-e5-0.6b) and I2_S (from bitnet-embeddings-0.6b) conversion process
 2026-07-12 04:10:30 +02:00
Yan Xia 01eb415772 Update demo link in README.md 2026-03-10 15:49:46 +08:00
tsong-ms 0fdaa16ae3 Merge pull request #421 from microsoft/fix/unsafe-deserialization-gpu-pipeline 
fix: add weights_only=True to torch.load in GPU inference pipeline
 2026-03-09 20:20:21 +08:00
UbuntuandClaude Opus 4.6 eb60fc39cb fix: add weights_only=True to torch.load in GPU inference pipeline 
Mitigate unsafe deserialization vulnerability (CWE-502) in the GPU
inference pipeline. torch.load without weights_only=True allows
arbitrary code execution via malicious pickle payloads in checkpoint
files.

Affected locations:
- gpu/convert_checkpoint.py:37 (checkpoint conversion utility)
- gpu/generate.py:67,69 (fp16 and int2 checkpoint loading)

The utils/ scripts already applied this parameter correctly; this
commit brings the GPU pipeline to the same safety standard.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
 2026-03-09 12:09:19 +00:00
tsong-ms 8fd3412fbc Merge pull request #406 from XsquirrelC/main 
[fix] convert pt to gguf
 2026-02-03 13:30:42 +08:00
XSquirrelC 3987a503fd [fix] convert pt to gguf 2026-02-03 05:24:24 +00:00
tsong-ms ade47a535c Merge pull request #380 from XsquirrelC/main 
[fix] change README link
 2026-01-27 13:48:51 +08:00
XSquirrelC 77e136fb08 [fix] change README link 2026-01-27 03:39:11 +00:00
tsong-ms cc3c9e4c80 Merge pull request #379 from XsquirrelC/main 
BitNet CPU Inference Optimization
 2026-01-27 11:24:02 +08:00
XSquirrelC 1876a3e889 [merge] submodule llama.cpp 2026-01-27 03:09:32 +00:00
XSquirrelC e8c8107dcf [modify] some test picture and add power test script 2026-01-25 06:51:33 +00:00
XSquirrelC 7b2c52b9d5 [modify] some utils test script 2026-01-24 08:40:36 +00:00
XSquirrelC 2fed9af730 [fix] setup_env.py bug COMPILER_EXTRA_ARGS 2026-01-22 11:11:14 +00:00
deva100 7e6f0e14f1 [modify] update README; [feat] some test script in utils 2026-01-22 06:33:03 +00:00
deva100 7ea1f2601f [modify] fine_tuning_result.png 2026-01-20 07:40:37 +00:00
deva100 b68802ff17 [fix] embed-quant q6_k; [modify] README update 2026-01-20 04:56:50 +00:00
deva100 35b1c28585 [fix] correct README 2026-01-15 03:44:50 +00:00
deva100 53ffe5e92b [chore] update README 2026-01-15 03:37:16 +00:00
deva100 43da5e5f76 [fix] make demo_benchmark.sh more fast 2025-12-23 07:23:14 +00:00
deva100 41cc304868 [chore] add some automation bash script for BitNet Tech Report 2025-12-23 06:48:33 +00:00
deva100 112f853414 [feat] I2S kernels for weight & activation parallel on Intel & ARM machine; [feat] I2S GEMV & GEMM(llama.cpp); [feat] quantize activation & dequantize embedding(llama.cpp); [fix] compile bug: cannot define __ARM_FEATURE_DOTPROD(llama.cpp) 2025-11-19 07:35:05 +00:00
Junhui He 404980eeca Merge pull request #290 from microsoft/gpu-readme-dev 
Update readme for gpu kernels
 2025-06-03 14:14:20 +08:00
Junhui He 088e607b25 Merge pull request #280 from microsoft/fix-convert-dev 
Enable conversion from .safetensors checkpoints to gguf files
 2025-06-03 13:59:47 +08:00
ZeonfaiHo c1e9a9a237 Update readme for gpu kernels 2025-05-31 21:41:41 +08:00
junhuihe 43e9b2d4a0 Enable conversion from .safetensors checkpoints to gguf files 2025-05-23 16:19:29 +08:00
tsong-ms 69a20459f5 Merge pull request #268 from younesbelkada/add-falcon-e-final 
Add falcon-e support
 2025-05-21 16:28:05 +08:00
younesbelkada 5c12850ed9 Merge branch 'add-falcon-e-final' of github.com:younesbelkada/BitNet into add-falcon-e-final 2025-05-21 11:53:40 +04:00
younesbelkada 765741d80b update submodule 2025-05-21 11:52:30 +04:00
Younes Belkada f314d18863 feat: add also base models 2025-05-21 04:11:07 +04:00
Younes Belkada 9e9575665e Merge branch 'microsoft:main' into add-falcon-e-final 2025-05-20 17:05:11 +04:00
tsong-ms 70285e0154 Merge pull request #276 from microsoft/readme-dev 
refine readme for gpu kernel
 2025-05-20 16:14:18 +08:00
tsong-ms 6197e9feb0 refine readme for gpu kernel 2025-05-20 12:29:56 +08:00
Junhui He 6c2c08f67e Merge pull request #266 from microsoft/gpu-dev 2025-05-19 12:46:20 +08:00
Junhui He 154c92b704 Init gpu branch 2025-05-19 04:34:00 +00:00
Younes Belkada 0015ad5201 Update README.md 2025-05-15 18:49:28 +04:00
younesbelkada de371b708d add falcon-e support 2025-05-14 17:07:05 +04:00
Benjamin Wegener c9e752c9d7 Fix build error with GCC by forcing Clang compiler in CMake on android/aarch64 (#242) 
GCC does not recognize Clang-specific warning flags like
-Wunreachable-code-break and -Wunreachable-code-return, which are passed
by upstream submodules (e.g., ggml). This patch forces CMake to use Clang
via command-line arguments, avoiding the need to patch nested submodules.

This resolves compiler errors without modifying submodule source code.
 2025-05-08 16:22:45 +08:00
Benjamin Wegener 1792346223 Add run_inference_server.py for Running llama.cpp Built-in Server (#204) 
* Update CMakeLists.txt

I added a CMake option to compile the Llama.cpp server. This update allows us to easily build and deploy the server using BitNet

* Create run_inference_server.py

same as run_inference, but for use with llama.cpp's built in server, for some extra comfort

In particular:
- The build directory is determined based on whether the system is running on Windows or not.
- A list of arguments (`--model`, `-m` etc.) is created.
- The main argument list is parsed and passed to the `subprocess.run()` method to execute the system command.
 2025-05-08 16:22:12 +08:00
Junhui He c17d1c5d77 Merge pull request #212 from microsoft/arch-name-dev 
Fix model architecture name
 2025-04-23 11:20:15 +08:00
junhuihe 488dc1e876 Fix model architecture name 2025-04-22 17:28:59 +08:00
tsong-ms fd9f1d6e46 Merge pull request #176 from microsoft/readme-dev 
refine readme
 2025-04-16 12:35:53 +08:00
tsong 874e6bd5fb refine readme 2025-04-16 04:34:59 +00:00
tsong-ms 034b34cb70 Merge pull request #175 from microsoft/readme-dev 
add third-party demo
 2025-04-15 22:42:12 +08:00
tsong 71fdd9472f add third-party demo 2025-04-15 14:36:05 +00:00