AIBOX-1684X-RAG 外挂知识库-运行ChatDoc-TPU出错
本帖最后由 bing20138 于 2024-6-25 09:16 编辑AIBOX-1684X运行ChatDoc-TPU出错(已经安装官方提供的sail sophon_arm-3.7.0-py3-none-any.whl)
步骤一、git clone https://github.com/wangyifan2018/ChatDoc-TPU.git下载工程
步骤二、./run.sh --model chatglm3 --dev_id 0下载大模型
步骤三、pip3 install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple安装依赖
步骤四、./run.sh --model chatglm3 --dev_id 0 运行出错。部分log显示如下
bm_memset_device failed, return=4
bm_alloc_gmem failed, dev_id = 0, size = 0x0
BM_CHECK_RET fail /workspace/libsophon/bmlib/src/bmlib_memory.cpp: bm_malloc_device_byte_heap_mask: 705
bm_malloc_device_type() err=5, size=0
bm_device_mem_range_valid saddr=0xfffffffff eaddr=0xfffffffff out of range
memset wrong memory addr 0xfffffffff
bm_memset_device failed, return=4
bm_alloc_gmem failed, dev_id = 0, size = 0x0
BM_CHECK_RET fail /workspace/libsophon/bmlib/src/bmlib_memory.cpp: bm_malloc_device_byte_heap_mask: 705
bm_malloc_device_type() err=5, size=0
bm_device_mem_range_valid saddr=0xfffffffff eaddr=0xfffffffff out of range
memset wrong memory addr 0xfffffffff
bm_memset_device failed, return=4
bm_alloc_gmem failed, dev_id = 0, size = 0x0
BM_CHECK_RET fail /workspace/libsophon/bmlib/src/bmlib_memory.cpp: bm_malloc_device_byte_heap_mask: 705
bm_malloc_device_type() err=5, size=0
bm_device_mem_range_valid saddr=0xfffffffff eaddr=0xfffffffff out of range
memset wrong memory addr 0xfffffffff
bm_memset_device failed, return=4
2024-06-25 08:56:20,643 - INFO - chatbot init success!
2024-06-25 08:58:00.294 Uncaught app exception
Traceback (most recent call last):
File "/usr/local/lib/python3.8/dist-packages/streamlit/runtime/scriptrunner/script_runner.py", line 552, in _run_script
exec(code, module.__dict__)
File "/home/linaro/ChatDoc-TPU/web_demo_st.py", line 178, in <module>
for result_answer, _ in chatbot_st.llm.stream_predict(user_input, his):
File "/home/linaro/ChatDoc-TPU/chat/chatglm3/chatglm3.py", line 269, in stream_predict
token = self.forward_first(tokens)
File "/home/linaro/ChatDoc-TPU/chat/chatglm3/chatglm3.py", line 147, in forward_first
input_ids = np.zeros(self.SEQLEN, type_convert(self.first_embed_input["dtype"]))
File "/home/linaro/ChatDoc-TPU/chat/utils.py", line 16, in type_convert
if sail_dtype == sail.Dtype.BM_FLOAT16:
AttributeError: type object 'sophon.sail.Dtype' has no attribute 'BM_FLOAT16'
可能是内存溢出了，尝试执行 sudo systemctl stop FireflyChat 先关闭 FireflyChat 的运行。
这个代码例程是芯片供应商提供的，后续运行上似乎有些问题。目前 FireflyChat 在开发 RAG 应用，也会提供开放的 API，下周应该就能在wiki上见到相应文档了。
页:
[1]