RK3568 PCIe 异常排查

firefly_zhongw

PCIe3.0控制器初始化设备系统异常

1. [   21.202444] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
   
2. [   21.202467] rcu:     3-...0: (1 GPs behind) idle=21a/1/0x4000000000000000 softirq=23/25 fqs=1856
   
3. [   21.202473] rcu:     (detected by 2, t=6302 jiffies, g=-1163, q=24)
   
4. [   21.202482] Task dump for CPU 3:
   
5. [   21.202488] rk-pcie         R  running task        0    54      2 0x0000002a
   
6. [   21.202497] Call trace:
   
7. [   21.202511]  __switch_to+0xe4/0x138
   
8. [   21.202518]  0x4
   
9. [   21.205777] rcu: INFO: rcu_sched detected stalls on CPUs/tasks:
   
10. [   21.205796] rcu:     3-...!: (24 ticks this GP) idle=21a/1/0x4000000000000000 softirq=5/25 fqs=1
    
11. [   21.205801] rcu:     (detected by 2, t=6302 jiffies, g=-1183, q=2)
    
12. [   21.205810] Task dump for CPU 3:
    
13. [   21.205815] rk-pcie         R  running task        0    54      2 0x0000002a
    
14. [   21.205823] Call trace:
    
15. [   21.205831]  __switch_to+0xe4/0x138
    
16. [   21.205837]  0x4
    
17. [   21.205846] rcu: rcu_sched kthread starved for 6299 jiffies! g-1183 f0x0 RCU_GP_WAIT_FQS(5) ->state=0x402 ->cpu=0
    
18. [   21.205851] rcu: RCU grace-period kthread stack dump:
    
19. [   21.205856] rcu_sched       I    0    11      2 0x00000028
    
20. [   21.205863] Call trace:
    
21. [   21.205869]  __switch_to+0xe4/0x138
    
22. [   21.205879]  __schedule+0x2f4/0x930
    
23. [   21.205886]  schedule+0x38/0xa0
    
24. [   21.205892]  schedule_timeout+0x194/0x478
    
25. [   21.205901]  rcu_gp_kthread+0x4fc/0x840
    
26. [   21.205909]  kthread+0x12c/0x158
    
27. [   21.205916]  ret_from_fork+0x10/0x18

复制代码

异常原因：
    如果系统卡住此 log 附近，则表明 PCIe3.0 的 PHY 工作异常。

解决方法：
硬件上使用PCIe
1. 外部晶振芯片的时钟输入是否异常,如果无时钟或者幅度异常,将导致 phy 无法锁定。
2. 检查 PCIE30_AVDD_0V9 和 PCIE30_AVDD_1V8 电压是否满足要求。

硬件上不使用PCIe
kernel 的 dts 里把 PCIe disabled。

1. &pcie30phy {
   
2.     status = "disabled";
   
3. };
   
4. 
   
5. &pcie3x2 {
   
6.     status = "disabled";
   
7. };
   

复制代码

liuwen1987

我们又遇到如下：
[    1.206302] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[    2.219630] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[    3.232964] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[    4.242959] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x1
[    5.256292] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[    6.269624] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[    7.286295] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[    8.296290] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[    9.309624] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[   10.326290] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[   11.336288] rk-pcie 3c0800000.pcie: PCIe Link Fail
[   11.336308] rk-pcie 3c0800000.pcie: failed to initialize host
[   21.592941] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[   21.592963] rcu:     2-...0: (2 GPs behind) idle=7c2/1/0x4000000000000000 softirq=57/59 fqs=1811
[   21.592969] rcu:     (detected by 0, t=6302 jiffies, g=-1039, q=289)
[   21.592978] Task dump for CPU 2:
[   21.592983] swapper/0       R  running task        0     1      0 0x0000002a
[   21.592992] Call trace:
[   21.593007]  __switch_to+0xe4/0x138
[   21.593025]  enqueue_task_fair+0x48c/0x1868
[   84.609593] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[   84.609615] rcu:     2-...0: (2 GPs behind) idle=7c2/1/0x4000000000000000 softirq=57/59 fqs=7335
[   84.609620] rcu:     (detected by 0, t=25207 jiffies, g=-1039, q=289)
[   84.609628] Task dump for CPU 2:
[   84.609633] swapper/0       R  running task        0     1      0 0x0000002a
[   84.609640] Call trace:
[   84.609649]  __switch_to+0xe4/0x138
[   84.609658]  enqueue_task_fair+0x48c/0x1868
[  147.626248] rcu: INFO: rcu_preempt detected stalls on CPUs/tasks:
[  147.626270] rcu:     2-...0: (2 GPs behind) idle=7c2/1/0x4000000000000000 softirq=57/59 fqs=12974
[  147.626275] rcu:     (detected by 1, t=44112 jiffies, g=-1039, q=289)
[  147.626283] Task dump for CPU 2:
[  147.626288] swapper/0       R  running task        0     1      0 0x0000002a
[  147.626295] Call trace:
[  147.626304]  __switch_to+0xe4/0x138
[  147.626312]  enqueue_task_fair+0x48c/0x1868，然后机卡死了，这个能办看下么

firefly_zhongw

自己做的底板，由于PCIe导致的问题，优先排查PCIe相关的硬件电路以及信号。

qingting11111

楼主，请问一下，您的问题解决了吗？遇到相似的问题

1786472307

[    6.299285] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x3
start rknn server, version:1.3.0 (121b661 build: 2022-04-29 11:11:47)
I NPUTransfer: Starting NPU Transfer Server, Transfer version 2.1.0 (b5861e7@2020-11-23T11:50:51)
[    7.312616] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x3
[    8.325947] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x3
[    9.339280] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x3
[   10.355946] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x3
[   11.365986] rk-pcie 3c0800000.pcie: PCIe Link Fail
[   11.366020] rk-pcie 3c0800000.pcie: failed to initialize host
我调试pcie也遇到了类似的错误，该如何解决呀

珂望5826

1786472307 发表于 2024-6-18 20:32
[    6.299285] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x3
start rknn server, version:1.3. ...

您好，解决了没？我也遇到这个问题了

1786472307

珂望5826 发表于 2024-6-19 11:40
您好，解决了没？我也遇到这个问题了

没有

喔亮晶晶

1786472307 发表于 2024-6-18 20:32
[    6.299285] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x3
start rknn server, version:1.3. ...

你好，这个问题解决了吗？

1786472307

喔亮晶晶 发表于 2024-6-29 16:25
你好，这个问题解决了吗？

硬件存在问题，重新还接一遍就解决了

^O^

liuwen1987 发表于 2021-9-1 19:04
我们又遇到如下：
[    1.206302] rk-pcie 3c0800000.pcie: PCIe Linking... LTSSM is 0x0
[    2.219630 ...

您好，请问题解决了嘛