AIBOX-1684X-RAG 外挂知识库-运行ChatDoc-TPU出错

bing20138

AIBOX-1684X运行ChatDoc-TPU出错(已经安装官方提供的sail sophon_arm-3.7.0-py3-none-any.whl[size=1.25em])
步骤一、git clone https://github.com/wangyifan2018/ChatDoc-TPU.git下载工程
步骤二、./run.sh --model chatglm3 --dev_id 0下载大模型
步骤三、pip3 install -r requirements.txt -i https://pypi.tuna.tsinghua.edu.cn/simple安装依赖
步骤四、./run.sh --model chatglm3 --dev_id 0 运行出错。部分log显示如下
[2024-06-25 08:56:20.527] [error] [tensor.cpp:403] bm_memset_device failed, return=4
[bmlib_memory][error] bm_alloc_gmem failed, dev_id = 0, size = 0x0
[BM_CHECK][error] BM_CHECK_RET fail /workspace/libsophon/bmlib/src/bmlib_memory.cpp: bm_malloc_device_byte_heap_mask: 705
[2024-06-25 08:56:20.565] [error] [tensor.cpp:396] bm_malloc_device_type() err=5, size=0
[bmlib_memory][error] bm_device_mem_range_valid saddr=0xfffffffff eaddr=0xfffffffff out of range
[bmlib_memory][error] memset wrong memory addr 0xfffffffff
[2024-06-25 08:56:20.565] [error] [tensor.cpp:403] bm_memset_device failed, return=4
[bmlib_memory][error] bm_alloc_gmem failed, dev_id = 0, size = 0x0
[BM_CHECK][error] BM_CHECK_RET fail /workspace/libsophon/bmlib/src/bmlib_memory.cpp: bm_malloc_device_byte_heap_mask: 705
[2024-06-25 08:56:20.603] [error] [tensor.cpp:396] bm_malloc_device_type() err=5, size=0
[bmlib_memory][error] bm_device_mem_range_valid saddr=0xfffffffff eaddr=0xfffffffff out of range
[bmlib_memory][error] memset wrong memory addr 0xfffffffff
[2024-06-25 08:56:20.604] [error] [tensor.cpp:403] bm_memset_device failed, return=4
[bmlib_memory][error] bm_alloc_gmem failed, dev_id = 0, size = 0x0
[BM_CHECK][error] BM_CHECK_RET fail /workspace/libsophon/bmlib/src/bmlib_memory.cpp: bm_malloc_device_byte_heap_mask: 705
[2024-06-25 08:56:20.642] [error] [tensor.cpp:396] bm_malloc_device_type() err=5, size=0
[bmlib_memory][error] bm_device_mem_range_valid saddr=0xfffffffff eaddr=0xfffffffff out of range
[bmlib_memory][error] memset wrong memory addr 0xfffffffff
[2024-06-25 08:56:20.642] [error] [tensor.cpp:403] bm_memset_device failed, return=4
2024-06-25 08:56:20,643 - INFO - chatbot init success!
2024-06-25 08:58:00.294 Uncaught app exception
Traceback (most recent call last):
  File "/usr/local/lib/python3.8/dist-packages/streamlit/runtime/scriptrunner/script_runner.py", line 552, in _run_script
    exec(code, module.__dict__)
  File "/home/linaro/ChatDoc-TPU/web_demo_st.py", line 178, in <module>
    for result_answer, _ in chatbot_st.llm.stream_predict(user_input, his):
  File "/home/linaro/ChatDoc-TPU/chat/chatglm3/chatglm3.py", line 269, in stream_predict
    token = self.forward_first(tokens)
  File "/home/linaro/ChatDoc-TPU/chat/chatglm3/chatglm3.py", line 147, in forward_first
    input_ids = np.zeros(self.SEQLEN, type_convert(self.first_embed_input["dtype"]))
  File "/home/linaro/ChatDoc-TPU/chat/utils.py", line 16, in type_convert
    if sail_dtype == sail.Dtype.BM_FLOAT16:
AttributeError: type object 'sophon.sail.Dtype' has no attribute 'BM_FLOAT16'

Multius

可能是内存溢出了，尝试执行 sudo systemctl stop FireflyChat 先关闭 FireflyChat 的运行。
这个代码例程是芯片供应商提供的，后续运行上似乎有些问题。目前 FireflyChat 在开发 RAG 应用，也会提供开放的 API，下周应该就能在wiki上见到相应文档了。