运行SOPHON-DEMO的Qwen出现内存报错
运行SOPHON-DEMO的Qwen出现内存报错步骤1、git clone https://github.com/sophgo/sophon-demo工程步骤2、cd Qwen目录
步骤3、先执行./scripts/download.sh下载模型
步骤4、pip3 install -r python/requirements.txt 和安装sophon_arm_pcie-3.7.0-py3-none-any.whl依赖
步骤5、streamlit run python/web_demo.py
错误显示-cmd终端显示：
bm_memset_device failed, return=4
bm_alloc_gmem failed, dev_id = 0, size = 0x0
BM_CHECK_RET fail /workspace/libsophon/bmlib/src/bmlib_memory.cpp: bm_malloc_device_byte_heap_mask: 705
bm_malloc_device_type() err=5, size=0
bm_device_mem_range_valid saddr=0xfffffffff eaddr=0xfffffffff out of range
memset wrong memory addr 0xfffffffff
bm_memset_device failed, return=4
bm_alloc_gmem failed, dev_id = 0, size = 0x0
BM_CHECK_RET fail /workspace/libsophon/bmlib/src/bmlib_memory.cpp: bm_malloc_device_byte_heap_mask: 705
bm_malloc_device_type() err=5, size=0
bm_device_mem_range_valid saddr=0xfffffffff eaddr=0xfffffffff out of range
memset wrong memory addr 0xfffffffff
bm_memset_device failed, return=4
bm_alloc_gmem failed, dev_id = 0, size = 0x0
BM_CHECK_RET fail /workspace/libsophon/bmlib/src/bmlib_memory.cpp: bm_malloc_device_byte_heap_mask: 705
bm_malloc_device_type() err=5, size=0
bm_device_mem_range_valid saddr=0xfffffffff eaddr=0xfffffffff out of range
memset wrong memory addr 0xfffffffff
bm_memset_device failed, return=4
bm_alloc_gmem failed, dev_id = 0, size = 0x0
BM_CHECK_RET fail /workspace/libsophon/bmlib/src/bmlib_memory.cpp: bm_malloc_device_byte_heap_mask: 705
bm_malloc_device_type() err=5, size=0
bm_device_mem_range_valid saddr=0xfffffffff eaddr=0xfffffffff out of range
memset wrong memory addr 0xfffffffff
bm_memset_device failed, return=4
bm_alloc_gmem failed, dev_id = 0, size = 0x0
BM_CHECK_RET fail /workspace/libsophon/bmlib/src/bmlib_memory.cpp: bm_malloc_device_byte_heap_mask: 705
bm_malloc_device_type() err=5, size=0
bm_device_mem_range_valid saddr=0xfffffffff eaddr=0xfffffffff out of range
memset wrong memory addr 0xfffffffff
bm_memset_device failed, return=4
错误显示-串口终端显示：
[ 3690.300620]ion_alloc failed(-22): len 0 heap_id_mask 4 flags 0
[ 3690.306888]ion_ioctl ion alloc failed, fd=-22, from streamlit
[ 3690.313185]ion_alloc failed(-22): len 0 heap_id_mask 1 flags 0
[ 3690.319452]ion_ioctl ion alloc failed, fd=-22, from streamlit
[ 3690.325633]ion_alloc failed(-22): len 0 heap_id_mask 2 flags 0
[ 3690.331897]ion_ioctl ion alloc failed, fd=-22, from streamlit
[ 3690.338074]ion_alloc failed(-22): len 0 heap_id_mask 4 flags 0
[ 3690.344337]ion_ioctl ion alloc failed, fd=-22, from streamlit
[ 3690.350833]ion_alloc failed(-22): len 0 heap_id_mask 1 flags 0
[ 3690.357103]ion_ioctl ion alloc failed, fd=-22, from streamlit
[ 3690.363282]ion_alloc failed(-22): len 0 heap_id_mask 2 flags 0
[ 3690.369548]ion_ioctl ion alloc failed, fd=-22, from streamlit
[ 3690.375725]ion_alloc failed(-22): len 0 heap_id_mask 4 flags 0
[ 3690.381988]ion_ioctl ion alloc failed, fd=-22, from streamlit
[ 3690.388292]ion_alloc failed(-22): len 0 heap_id_mask 1 flags 0
[ 3690.394554]ion_ioctl ion alloc failed, fd=-22, from streamlit
[ 3690.400743]ion_alloc failed(-22): len 0 heap_id_mask 2 flags 0
[ 3690.407011]ion_ioctl ion alloc failed, fd=-22, from streamlit
[ 3690.413193]ion_alloc failed(-22): len 0 heap_id_mask 4 flags 0
[ 3690.419463]ion_ioctl ion alloc failed, fd=-22, from streamlit
[ 3690.425752]ion_alloc failed(-22): len 0 heap_id_mask 1 flags 0
[ 3690.432020]ion_ioctl ion alloc failed, fd=-22, from streamlit
[ 3690.438205]ion_alloc failed(-22): len 0 heap_id_mask 2 flags 0
[ 3690.444486]ion_ioctl ion alloc failed, fd=-22, from streamlit
[ 3690.450671]ion_alloc failed(-22): len 0 heap_id_mask 4 flags 0
[ 3690.456941]ion_ioctl ion alloc failed, fd=-22, from streamlit
可能是 FireflyChat 占用了内存导致的，执行 `sudo systemctl stop FireflyChat` 命令关闭 FireflyChat 后再继续尝试运行 demo Multius 发表于 2024-6-17 15:11
可能是 FireflyChat 占用了内存导致的，执行 `sudo systemctl stop FireflyChat` 命令关闭 FireflyChat 后 ...
试过还是不行 bing20138 发表于 2024-6-17 15:33
试过还是不行
使用的具体是 Qwen 的哪一个模型？模型最终编译出来的 bmodel 文件有多大体积？ 也试过./memory_edit.sh -c -npu 7615 -vpu 3072 -vpp 3072修改TPU内存也是不行的 Multius 发表于 2024-6-17 15:52
使用的具体是 Qwen 的哪一个模型？模型最终编译出来的 bmodel 文件有多大体积？
qwen-7b_int4_1dev.bmodel 4.78GB
难道是因为这个原因？ bing20138 发表于 2024-6-17 16:06
难道是因为这个原因？
这里的说明是对于编译模型的机器的要求，与模型运行无关 bing20138 发表于 2024-6-17 16:02
qwen-7b_int4_1dev.bmodel 4.78GB
这个尺寸的 int4 的量化模型应该是足够在 1684x 上面运行的，但由于 Qwen 第一代模型的适配工作也有些时日了，具体原因难以排查。
建议使用 Qwen1.5 或者 Qwen2 进行部署验证。
Firefly | 让科技更简单，让生活更智能 (t-firefly.com)，我们在这里提供了 Qwen1.5 编译好的 bmodel 文件，可直接使用。
换QWEN1.5会报AttributeError: module 'sophon.sail' has no attribute 'EngineLLM'
页:
[1]
2