Firefly开源社区

标题: 加载xilinx的xdma驱动出错 [打印本页]
作者: rudy_jenny    时间: 2022-6-24 17:41
标题: 加载xilinx的xdma驱动出错
Ubuntu系统，出错log如下：
interrupt_selection 1.
Loading driver...insmod xdma.ko interrupt_mode=2 ...
[   79.039826] xdma:probe_one: pdev 0x00000000805013c7, err -22.

Error: The Kernel module installed correctly, but no devices were recognized.
FAILED

作者: 799959745    时间: 2022-6-27 14:32
提供相应的ko文件，更多的log帮助分析复现
作者: rudy_jenny    时间: 2022-6-27 15:21
799959745 发表于 2022-6-27 14:32
提供相应的ko文件，更多的log帮助分析复现

1、加载驱动命令
./load_driver.sh
打印如下：Loading driver...
Error: The Kernel module installed correctly, but no devices were recognized.
FAILED

2、列举PCIe设备信息命令
lspci
打印如下：0000:00:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 3588 (rev 01)
0000:01:00.0 Memory controller: Xilinx Corporation Device 7024
0002:20:00.0 PCI bridge: Fuzhou Rockchip Electronics Co., Ltd Device 3588 (rev 01)
0002:21:00.0 Network controller: Broadcom Inc. and subsidiaries Device 449d (rev 02)

xdma.zip

251.17 KB, 下载次数: 23, 下载积分: 灯泡 -1 , 经验 -1

xdma驱动

Xilinx_Answer_65444_Linux_Files.zip

472.84 KB, 下载次数: 12, 下载积分: 灯泡 -1 , 经验 -1

xdma源码
作者: 799959745    时间: 2022-6-28 10:15
rudy_jenny 发表于 2022-6-27 15:21
1、加载驱动命令
./load_driver.sh
打印如下：Loading driver...

按照log来说驱动加载并没有问题。只是没有识别到pcie设备
作者: rudy_jenny    时间: 2022-6-28 10:43
799959745 发表于 2022-6-27 14:32
提供相应的ko文件，更多的log帮助分析复现

以下是dmesg中的pci信息，怀疑是不是能够给其他PCIe设备分配的内存不够导致的。
[    2.447142] vcc3v3_pcie30: 3300 mV, disabled
[    2.447188] reg-fixed-voltage vcc3v3-pcie30: Looking up vin-supply from device tree
[    2.447191] vcc3v3_pcie30: supplied by vcc12v_dcin
[    2.447225] reg-fixed-voltage vcc3v3-pcie30: vcc3v3_pcie30 supplying 3300000uV
[    2.994201] rk-pcie fe150000.pcie: invalid prsnt-gpios property in node
[    2.994225] rk-pcie fe150000.pcie: Looking up vpcie3v3-supply from device tree
[    2.994245] rk-pcie fe170000.pcie: invalid prsnt-gpios property in node
[    2.994253] rk-pcie fe170000.pcie: Looking up vpcie3v3-supply from device tree
[    2.994262] rk-pcie fe170000.pcie: Looking up vpcie3v3-supply property in node /pcie@fe170000 failed
[    2.994268] rk-pcie fe170000.pcie: no vpcie3v3 regulator found
[    2.994451] rk-pcie fe170000.pcie: missing legacy IRQ resource
[    2.994461] rk-pcie fe170000.pcie: IRQ msi not found
[    2.994465] rk-pcie fe170000.pcie: use outband MSI support
[    2.994469] rk-pcie fe170000.pcie: Missing *config* reg space
[    2.994477] rk-pcie fe170000.pcie: host bridge /pcie@fe170000 ranges:
[    2.994490] rk-pcie fe170000.pcie:      err 0x00f2000000..0x00f20fffff -> 0x00f2000000
[    2.994499] rk-pcie fe170000.pcie:       IO 0x00f2100000..0x00f21fffff -> 0x00f2100000
[    2.994508] rk-pcie fe170000.pcie:      MEM 0x00f2200000..0x00f2ffffff -> 0x00f2200000
[    2.994516] rk-pcie fe170000.pcie:      MEM 0x0980000000..0x09bfffffff -> 0x0980000000
[    2.994539] rk-pcie fe170000.pcie: Missing *config* reg space
[    2.994564] rk-pcie fe170000.pcie: invalid resource
[    3.010838] rk-pcie fe150000.pcie: missing legacy IRQ resource
[    3.010858] rk-pcie fe150000.pcie: IRQ msi not found
[    3.010868] rk-pcie fe150000.pcie: use outband MSI support
[    3.010878] rk-pcie fe150000.pcie: Missing *config* reg space
[    3.010895] rk-pcie fe150000.pcie: host bridge /pcie@fe150000 ranges:
[    3.010922] rk-pcie fe150000.pcie:      err 0x00f0000000..0x00f00fffff -> 0x00f0000000
[    3.010949] rk-pcie fe150000.pcie:       IO 0x00f0100000..0x00f01fffff -> 0x00f0100000
[    3.010977] rk-pcie fe150000.pcie:      MEM 0x00f0200000..0x00f0ffffff -> 0x00f0200000
[    3.010995] rk-pcie fe150000.pcie:      MEM 0x0900000000..0x093fffffff -> 0x0900000000
[    3.011033] rk-pcie fe150000.pcie: Missing *config* reg space
[    3.011066] rk-pcie fe150000.pcie: invalid resource
[    3.199139] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[    3.224695] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[    3.251359] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[    3.271387] rk-pcie fe150000.pcie: PCIe Link up, LTSSM is 0x130011
[    3.271579] rk-pcie fe150000.pcie: PCI host bridge to bus 0000:00
[    3.271599] pci_bus 0000:00: root bus resource [bus 00-0f]
[    3.271614] pci_bus 0000:00: root bus resource [??? 0xf0000000-0xf00fffff flags 0x0]
[    3.271631] pci_bus 0000:00: root bus resource [io  0x100000-0x1fffff] (bus address [0xf0100000-0xf01fffff])
[    3.271644] pci_bus 0000:00: root bus resource [mem 0xf0200000-0xf0ffffff]
[    3.271657] pci_bus 0000:00: root bus resource [mem 0x900000000-0x93fffffff pref]
[    3.271707] pci 0000:00:00.0: [1d87:3588] type 01 class 0x060400
[    3.271732] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x3fffffff]
[    3.271750] pci 0000:00:00.0: reg 0x14: [mem 0x00000000-0x3fffffff]
[    3.271767] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
[    3.271848] pci 0000:00:00.0: supports D1 D2
[    3.271862] pci 0000:00:00.0: PME# supported from D0 D1 D3hot
[    3.278024] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[    3.282362] pci_bus 0000:01: busn_res: can not insert [bus 01-ff] under [bus 00-0f] (conflicts with (null) [bus 00-0f])
[    3.282473] pci 0000:01:00.0: [10ee:7024] type 00 class 0x058000
[    3.282541] pci 0000:01:00.0: reg 0x10: [mem 0x00000000-0x01ffffff]
[    3.282578] pci 0000:01:00.0: reg 0x14: [mem 0x00000000-0x0000ffff]
[    3.282615] pci 0000:01:00.0: reg 0x18: [mem 0x00000000-0x01ffffff]
[    3.282744] pci 0000:01:00.0: enabling Extended Tags
[    3.282950] pci 0000:01:00.0: PME# supported from D0 D1 D2 D3hot
[    3.293278] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]
[    3.293308] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]
[    3.293324] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]
[    3.293337] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]
[    3.293351] pci 0000:00:00.0: BAR 8: no space for [mem size 0x05000000]
[    3.293364] pci 0000:00:00.0: BAR 8: failed to assign [mem size 0x05000000]
[    3.293380] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0200000-0xf020ffff pref]
[    3.293400] pci 0000:01:00.0: BAR 0: no space for [mem size 0x02000000]
[    3.293413] pci 0000:01:00.0: BAR 0: failed to assign [mem size 0x02000000]
[    3.293426] pci 0000:01:00.0: BAR 2: no space for [mem size 0x02000000]
[    3.293438] pci 0000:01:00.0: BAR 2: failed to assign [mem size 0x02000000]
[    3.293451] pci 0000:01:00.0: BAR 1: no space for [mem size 0x00010000]
[    3.293463] pci 0000:01:00.0: BAR 1: failed to assign [mem size 0x00010000]
[    3.293477] pci 0000:00:00.0: PCI bridge to [bus 01-ff]
[    3.296775] pcieport 0000:00:00.0: PME: Signaling with IRQ 142
[    3.304692] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[    3.331358] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[    3.358027] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[    3.374942] ehci-pci: EHCI PCI platform driver
[    3.384691] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[    3.411358] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[    3.438023] rk-pcie fe170000.pcie: PCIe Linking... LTSSM is 0x3
[    3.619781] reg-fixed-voltage pcie30-avdd1v8: Looking up vin-supply from device tree
[    3.619863] pcie30_avdd1v8: supplied by avcc_1v8_s0
[    3.620091] pcie30_avdd1v8: 1800 mV, enabled
[    3.620264] reg-fixed-voltage pcie30-avdd1v8: pcie30_avdd1v8 supplying 1800000uV
[    3.620383] reg-fixed-voltage pcie30-avdd0v75: Looking up vin-supply from device tree
[    3.620435] pcie30_avdd0v75: supplied by avdd_0v75_s0
[    3.620589] pcie30_avdd0v75: 750 mV, enabled
[    3.620744] reg-fixed-voltage pcie30-avdd0v75: pcie30_avdd0v75 supplying 750000uV
[    3.654754] rk-pcie fe170000.pcie: PCIe Link up, LTSSM is 0x30011
[    3.654970] rk-pcie fe170000.pcie: PCI host bridge to bus 0002:20
[    3.654990] pci_bus 0002:20: root bus resource [bus 20-2f]
[    3.655005] pci_bus 0002:20: root bus resource [??? 0xf2000000-0xf20fffff flags 0x0]
[    3.655021] pci_bus 0002:20: root bus resource [io  0x0000-0xfffff] (bus address [0xf2100000-0xf21fffff])
[    3.655035] pci_bus 0002:20: root bus resource [mem 0xf2200000-0xf2ffffff]
[    3.655051] pci_bus 0002:20: root bus resource [mem 0x980000000-0x9bfffffff pref]
[    3.655100] pci 0002:20:00.0: [1d87:3588] type 01 class 0x060400
[    3.655141] pci 0002:20:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref]
[    3.655335] pci 0002:20:00.0: supports D1 D2
[    3.655351] pci 0002:20:00.0: PME# supported from D0 D1 D3hot
[    3.658739] pci 0002:20:00.0: Primary bus is hard wired to 0
[    3.658744] pci 0002:20:00.0: bridge configuration invalid ([bus 01-ff]), reconfiguring
[    3.658884] pci 0002:21:00.0: [14e4:449d] type 00 class 0x028000
[    3.658956] pci 0002:21:00.0: reg 0x10: [mem 0x00000000-0x0000ffff 64bit]
[    3.659002] pci 0002:21:00.0: reg 0x18: [mem 0x00000000-0x003fffff 64bit]
[    3.659449] pci 0002:21:00.0: supports D1 D2
[    3.659452] pci 0002:21:00.0: PME# supported from D0 D1 D2 D3hot D3cold
[    3.675040] pci_bus 0002:21: busn_res: [bus 21-2f] end is updated to 21
[    3.675074] pci 0002:20:00.0: BAR 8: assigned [mem 0xf2200000-0xf27fffff]
[    3.675086] pci 0002:20:00.0: BAR 6: assigned [mem 0xf2800000-0xf280ffff pref]
[    3.675099] pci 0002:21:00.0: BAR 2: assigned [mem 0xf2400000-0xf27fffff 64bit]
[    3.675148] pci 0002:21:00.0: BAR 0: assigned [mem 0xf2200000-0xf220ffff 64bit]
[    3.675193] pci 0002:20:00.0: PCI bridge to [bus 21]
[    3.675203] pci 0002:20:00.0:   bridge window [mem 0xf2200000-0xf27fffff]
[    3.677404] pcieport 0002:20:00.0: PME: Signaling with IRQ 159
[    7.455652] [dhd] dhdpcie_pci_probe : no mutex held. set lock
[    7.455656] [dhd] dhdpcie_init: found adapter info 'DHD generic adapter'
[    7.455712] pcieh 0002:21:00.0: enabling device (0000 -> 0002)
[    7.533366] [dhd] dhdpcie_bar1_window_switch_enab: bar1_switch_enab=0 ramstart=0x170000 ramend=0x2affff bar1_size=0x0
[    7.580221] [dhd] dhdpcie_bus_attach: making DHD_BUS_DOWN
[    7.580231] [dhd] dhdpcie_init: rc_dev from dev->bus->self (1d87:3588) is ffffff8005fa0000
[    7.580339] [dhd] dhdpcie_init: rc_ep_aspm_cap: 1 rc_ep_l1ss_cap: 1
[    7.580351] [dhd] dhdpcie_request_irq: INTx enabled
[    7.580405] [dhd] dhdpcie_bar1_window_switch_enab: bar1_switch_enab=0 ramstart=0x170000 ramend=0x2affff bar1_size=0x400000
[    7.581000] [dhd] dhdpcie_oob_intr_unregister: irq is not registered
[    7.688040] [dhd] dhdpcie_pci_probe : the lock is released.
[    8.971510] [dhd] dhd_bus_devreset: dhdpcie_bus_start_host_dev OK
[    8.972807] [dhd] dhdpcie_bar1_window_switch_enab: bar1_switch_enab=0 ramstart=0x170000 ramend=0x2affff bar1_size=0x400000
[    8.972828] [dhd] dhdpcie_request_irq: INTx enabled
[    8.972868] [dhd] dhdpcie_dump_resource: BAR0(VA): 0x0000000000000000, BAR0(PA): 0x00000000f2200000, SIZE: 32768
[    8.972873] [dhd] dhdpcie_dump_resource: BAR1(VA): 0x0000000000000000, BAR1(PA): 0x00000000f2400000, SIZE: 4194304
[    8.974102] [dhd] dhd_conf_set_path_params : Final fw_path=/vendor/etc/firmware/fw_bcm43752a2_pcie_ag.bin
[    8.974116] [dhd] dhd_conf_set_path_params : Final clm_path=/vendor/etc/firmware/clm_bcm43752a2_pcie_ag.blob
[    8.974208] [dhd] dhdpcie_download_code_file: dhd_tcm_test_enable 0
[    8.974213] [dhd] dhdpcie_download_code_file: download firmware /vendor/etc/firmware/fw_bcm43752a2_pcie_ag.bin
[    8.974272] [dhd] dhd_os_open_image1: /vendor/etc/firmware/fw_bcm43752a2_pcie_ag.bin (857142 bytes) open success
[    9.062195] [dhd] dhdpcie_download_nvram: dhd_get_download_buffer len 7335
[    9.062221] [dhd] dhdpcie_download_nvram: process_nvram_vars len 5860
[    9.079980] [dhd] dhdpcie_bus_write_vars: Download, Upload and compare of NVRAM succeeded.
[    9.160382] [dhd] dhdpcie_readshared: PCIe shared addr (0x001f8974) read took 80000 usec before dongle is ready
[    9.160837] [dhd] dhdpcie_readshared: max H2D queues 40
[    9.160878] [dhd] dhdpcie_oob_intr_register OOB irq=161 flags=0x1
[    9.160919] [dhd] dhdpcie_oob_intr_register: enable_irq_wake
[    9.161035] [dhd] dhd_bus_dump_dar_registers: dar_errlog(0xa60:0x0) dar_erraddr(0xa64:0x0) dar_pcie_mbint(0xa68:0x0)
[    9.190336] [dhd] dhd_os_open_image1: /vendor/etc/firmware/clm_bcm43752a2_pcie_ag.blob (29225 bytes) open success

作者: rudy_jenny    时间: 2022-6-29 08:49
问题已解决
作者: Liuth    时间: 2022-6-29 09:04
rudy_jenny 发表于 2022-6-29 08:49
问题已解决

解决方法可以说一下吗？
作者: rudy_jenny    时间: 2022-6-29 11:27
Liuth 发表于 2022-6-29 09:04
解决方法可以说一下吗？

pci 0000:01:00.0: BAR 2: no space for [mem size 0x02000000]，看这一句，主要是rk3588的pcie低端bar内存大小只有16MB，对端FPGA把内存配置减小就好了
作者: wx__BkXu5u    时间: 2022-8-2 10:41
请问您做的是itx 3588j和fpga板卡通过pcie互联嘛，我想咨询一下3588开发板的pcie的endpoint模式切换能不能实现，通过rk3588官方的修改dts就可以吗
作者: rudy_jenny    时间: 2022-8-2 11:02
wx__BkXu5u 发表于 2022-8-2 10:41
请问您做的是itx 3588j和fpga板卡通过pcie互联嘛，我想咨询一下3588开发板的pcie的endpoint模式切换能不能 ...

dts没有修改，主要是FPGA那边进行适配，你要看一下3588的dts文件，最大能支持多少的内存
作者: wx__BkXu5u    时间: 2022-8-5 16:27
rudy_jenny 发表于 2022-8-2 11:02
dts没有修改，主要是FPGA那边进行适配，你要看一下3588的dts文件，最大能支持多少的内存

好的，谢谢啦
作者: chaoloveai120    时间: 2023-1-11 14:23
谢谢分享解决经验
作者: xiaguangbo    时间: 2023-6-30 11:19
我也出现了同样的报错，lspci能看到设备，就是报 bar 空间不够。我用的是 riffa, 所以 fpga 的 pcie ip 用的是最基础那个，也有 bar 设置, 建议改哪个？
[attach]12662[/attach]
作者: xiaguangbo    时间: 2023-6-30 11:35
我看了下，我这里 fpga 所在的 pci 01:00.0 是分配了的，1K*32b 正好是 0x3ff，但是 pci 00:00.0 有问题，这各是rk3588自己的东西吧，影响吗？

  1. debian@debian:~$ lspci                                                         
  2. 00:00.0 PCI bridge: Rockchip Electronics Co., Ltd RK3588 (rev 01)               
  3. 01:00.0 Memory controller: Xilinx Corporation Device 7024
复制代码

  1. [    3.041940] pci 0000:00:00.0: BAR 0: no space for [mem size 0x40000000]      
  2. [    3.041945] pci 0000:00:00.0: BAR 0: failed to assign [mem size 0x40000000]  
  3. [    3.041950] pci 0000:00:00.0: BAR 1: no space for [mem size 0x40000000]      
  4. [    3.041954] pci 0000:00:00.0: BAR 1: failed to assign [mem size 0x40000000]  
  5. [    3.041958] pci 0000:00:00.0: BAR 8: assigned [mem 0xf0200000-0xf02fffff]   
  6. [    3.041963] pci 0000:00:00.0: BAR 6: assigned [mem 0xf0300000-0xf030ffff pref
  7. ]                                                                              
  8. [    3.041969] pci 0000:01:00.0: BAR 0: assigned [mem 0xf0200000-0xf02003ff]   
  9. [    3.041982] pci 0000:00:00.0: PCI bridge to [bus 01-ff]                     
  10. [    3.041987] pci 0000:00:00.0:   bridge window [mem 0xf0200000-0xf02fffff]
复制代码

image.png (42.34 KB, 下载次数: )

image.png
作者: wx__H6Z50O    时间: 2023-7-19 14:38
rudy_jenny 发表于 2022-6-29 11:27
pci 0000:01:00.0: BAR 2: no space for [mem size 0x02000000]，看这一句，主要是rk3588的pcie低端bar内 ...

麻烦问下具体如何去改呢。 我尝试修改dma_pcie的pcie:Bars中的size为1或其他。还是会报no space for的错误
作者: Jack700    时间: 2024-3-27 16:33
好好学习一下，不知道后面如何XDMA
作者: mmry    时间: 2024-5-6 19:40
你好，请问我在使用3588编译xdma.ko的时候会报错误，如图mmiowb()这个函数找不到，请问是什么原因如何解决呢？谢谢！

微信图片_20240506193818.jpg (307.88 KB, 下载次数: 268)

微信图片_20240506193818.jpg
作者: mmry    时间: 2024-5-6 19:48
rudy_jenny 发表于 2022-6-29 11:27
pci 0000:01:00.0: BAR 2: no space for [mem size 0x02000000]，看这一句，主要是rk3588的pcie低端bar内 ...

请问这个是改的FPGA的哪里呢？
作者: wx__gIrM6i    时间: 2024-5-19 14:23
遇到类似问题，来看下



欢迎光临 Firefly开源社区 (https://dev.t-firefly.com/) Powered by Discuz! X3.1