Skip to content

Commit 1eb99eb

Browse files
Doc update for using INC for GPTQ INT4 (#3485)
* Doc update for using INC for GPTQ INT4 * update per comment * remove instruction for deprecated example * update DeepSpeed parts * cherry-pick #3486 * correct BKC for accuracy testing with deepspeed --------- Co-authored-by: Chunyuan WU <[email protected]>
1 parent 09be4c9 commit 1eb99eb

File tree

6 files changed

+147
-566
lines changed

6 files changed

+147
-566
lines changed

docker/Dockerfile.prebuilt

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -40,7 +40,8 @@ ARG TORCHCCL_VERSION=2.6.0
4040
ARG PYTORCH_VERSION=2.6.0
4141
ARG TORCHAUDIO_VERSION=2.6.0
4242
ARG TORCHVISION_VERSION=0.21.0
43-
RUN . ./venv/bin/activate && \
43+
RUN python -m venv venv && \
44+
. ./venv/bin/activate && \
4445
python -m pip --no-cache-dir install --upgrade \
4546
pip \
4647
setuptools \

examples/cpu/inference/python/llm-modeling/README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -72,7 +72,7 @@ conda install gperftools -y
7272
python -m pip install transformers==4.38.1 accelerate
7373

7474
# Set the environment variables for performance on Xeon
75-
export LD_PRELOAD=$(bash ../../../llm/tools/get_libstdcpp_lib.sh):${CONDA_PREFIX}/lib/libiomp.so:${CONDA_PREFIX}/lib/libtcmalloc.so:${LD_PRELOAD}
75+
export LD_PRELOAD=$(bash ../../../llm/tools/get_libstdcpp_lib.sh):${CONDA_PREFIX}/lib/libiomp5.so:${CONDA_PREFIX}/lib/libtcmalloc.so:${LD_PRELOAD}
7676
export KMP_BLOCKTIME=1
7777
export KMP_TPAUSE=0
7878
export KMP_FORKJOIN_BARRIER_PATTERN=dist,dist

0 commit comments

Comments
 (0)