lee/BitNet
1
0
Fork 0
mirror of https://github.com/microsoft/BitNet.git synced 2026-08-23 09:26:33 +00:00
Code Issues Packages Projects Releases Wiki Activity

Fix inference pipeline: remove hardcoded batch size, enable tool builds, update Python requirement

Browse Source
This commit is contained in:
isHuangXin
2026-07-15 11:59:58 +02:00
parent 425fbbcb75
commit 403efc7b00
3 changed files with 3 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+2 -2
View File
@@ -159,7 +159,7 @@ This project is based on the [llama.cpp](https://github.com/ggerganov/llama.cpp)
## Installation
### Requirements
- python>=3.9
- python>=3.10
- cmake>=3.22
- clang>=18
- For Windows users, install [Visual Studio 2022](https://visualstudio.microsoft.com/downloads/). In the installer, toggle on at least the following options(this also automatically installs the required additional tools like CMake):
@@ -186,7 +186,7 @@ cd BitNet
2. Install the dependencies
```bash
# (Recommended) Create a new conda environment
conda create -n bitnet-cpp python=3.9
conda create -n bitnet-cpp python=3.10
conda activate bitnet-cpp
pip install -r requirements.txt
run_inference.py
-1
View File
@@ -30,7 +30,6 @@ def run_inference():
'-ngl', '0',
'-c', str(args.ctx_size),
'--temp', str(args.temperature),
"-b", "1",
]
if args.conversation:
command.append("-cnv")
setup_env.py
+1 -1
View File
@@ -211,7 +211,7 @@ def compile():
logging.error(f"Arch {arch} is not supported yet")
exit(0)
logging.info("Compiling the code using CMake.")
run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), []), "-DCMAKE_C_COMPILER=clang", "-DCMAKE_CXX_COMPILER=clang++"], log_step="generate_build_files")
run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), []), "-DCMAKE_C_COMPILER=clang", "-DCMAKE_CXX_COMPILER=clang++", "-DLLAMA_BUILD_TOOLS=ON", "-DLLAMA_BUILD_EXAMPLES=ON"], log_step="generate_build_files")
# run_command(["cmake", "--build", "build", "--target", "llama-cli", "--config", "Release"])
run_command(["cmake", "--build", "build", "--config", "Release"], log_step="compile")