I have been trying to diagnose an issue the last few days in regards to CM4 modules freezing sometimes during a soft reboot, specifically when this certain pci wifi card is installed. I have been able to replicate it on multiple CM4 modules on different IO boards (the official cm4 io board as well as the Waveshare dual ethernet wifi base board) and using multiple OSs (Raspbian and OpenWRT). I have also tried different power supplies, including a benchtop BK Precision 9117 to make sure power was not the problem. I also tried booting from the internal emmc and an external usb drive. The freezing happens only maybe 10-20% of the time and otherwise isn't an issue. I've tried "delay_boot=1" and even "delay_boot=10" in config.txt.
Specific WiFi card in question: https://asiarf.com/product/wi-fi-6-11ax ... w7915-np1/
In the code block below, if is reboots properly, you see the "[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]" but if it hangs, everything in the code block will appear EXCEPT for "[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]" and it will stay that way indefinitely.
At this point, I don't know what the next steps in troubleshooting would be. I have also updated the bootloader to release 2024-07-30. To increase debug output, i've also added the following things (not sure which ones work vs don't work, but I get a bunch of debug messages). I've also added as much as I think someone could want to know down in the bottom.
config.txt
uart_2ndstage=1
disable_splash=1
dtdebug=1
enable_uart=1
cmdline.txt
loglevel=15 earlyprintk=serial,serial0,115200 debug initcall_debug
vcgencmd versionvcgencmd bootloader_versionvcgencmd bootloader_configuname -acat /boot/config.txt cat /boot/cmdline.txtlspci -vvv
Specific WiFi card in question: https://asiarf.com/product/wi-fi-6-11ax ... w7915-np1/
In the code block below, if is reboots properly, you see the "[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]" but if it hangs, everything in the code block will appear EXCEPT for "[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]" and it will stay that way indefinitely.
At this point, I don't know what the next steps in troubleshooting would be. I have also updated the bootloader to release 2024-07-30. To increase debug output, i've also added the following things (not sure which ones work vs don't work, but I get a bunch of debug messages). I've also added as much as I think someone could want to know down in the bottom.
config.txt
uart_2ndstage=1
disable_splash=1
dtdebug=1
enable_uart=1
cmdline.txt
loglevel=15 earlyprintk=serial,serial0,115200 debug initcall_debug
Code:
2.44 [sdcard] pieeprom.upd not found 2.45 [sdcard] recover4.elf not found 2.48 [sdcard] recovery.elf not found 2.72 Read start4.elf bytes 2248864 hnd 0x18ac 2.75 Read fixup4.dat bytes 5394 hnd 0x2438 2.78 0x00a03141 0x00000000 0x00001fff 2.82 MEM GPU: 128 ARM: 896 TOTAL: 1024 2.14 Starting start4.elf @ 0xfec00200 partition 0 2.16 PCI0 reset 2.27 +MESS:00:00:02.635600:0: arasan: arasan_emmc_openMESS:00:00:02.838380:0: brfs: File read: /mfs/sd/config.txtMESS:00:00:02.841003:0: brfs: File read: 800 bytesMESS:00:00:02.845835:0: brfs: File read: /mfs/sd/distroconfig.txtMESS:00:00:02.851796:0: brfs: File read: 638 bytesMESS:00:00:02.908545:0: HDMI1:EDID error reading EDID block 0 attempt 0MESS:00:00:02.913046:0: HDMI1:EDID giving up on reading EDID block 0MESS:00:00:02.918814:0: brfs: File read: /mfs/sd/config.txtMESS:00:00:02.923583:0: brfs: File read: 800 bytesMESS:00:00:02.928416:0: brfs: File read: /mfs/sd/distroconfig.txtMESS:00:00:03.934323:0: gpioman: gpioman_get_pin_num: pin DISPLAY_SDA not definedMESS:00:00:03.938701:0: gpioman: gpioman_get_pin_num: pin LEDS_PWR_OK not definedMESS:00:00:03.957832:0: gpioman: gpioman_get_pin_num: pin FLASH_0_ENABLE not definedMESS:00:00:03.962470:0: gpioman: gpioman_get_pin_num: pin FLASH_0_INDICATOR not definedMESS:00:00:03.970211:0: gpioman: gpioman_get_pin_num: pin FLASH_0_ENABLE not definedMESS:00:00:03.977662:0: gpioman: gpioman_get_pin_num: pin FLASH_0_INDICATOR not definedMESS:00:00:04.434694:0: gpioman: gpioman_get_pin_num: pin LEDS_PWR_OK not definedMESS:00:00:04.440341:0: *** Restart loggingMESS:00:00:04.442982:0: brfs: File read: 638 bytesMESS:00:00:04.485292:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 0MESS:00:00:04.490323:0: hdmi: HDMI1:EDID giving up on reading EDID block 0MESS:00:00:04.503468:0: hdmi: HDMI1:EDID error reading EDID block 0 attempt 0MESS:00:00:04.510362:0: hdmi: HDMI1:EDID giving up on reading EDID block 0MESS:00:00:04.514186:0: hdmi: HDMI:hdmi_get_state is deprecated, use hdmi_get_display_state insteadMESS:00:00:04.522940:0: HDMI0: hdmi_pixel_encoding: 300000000MESS:00:00:04.528419:0: HDMI1: hdmi_pixel_encoding: 300000000MESS:00:00:04.535233:0: gpioman: gpioman_get_pin_num: pin CAMERA_0_I2C_PORT not definedMESS:00:00:04.546040:0: dtb_file 'bcm2711-rpi-cm4.dtb'MESS:00:00:04.559099:0: brfs: File read: /mfs/sd/bcm2711-rpi-cm4.dtbMESS:00:00:04.562393:0: Loading 'bcm2711-rpi-cm4.dtb' to 0x100 size 0xce98MESS:00:00:04.602544:0: brfs: File read: 52888 bytesMESS:00:00:04.799449:0: brfs: File read: /mfs/sd/config.txtMESS:00:00:04.802241:0: brfs: File read: 800 bytesMESS:00:00:04.807139:0: brfs: File read: /mfs/sd/distroconfig.txtMESS:00:00:04.812641:0: brfs: File read: 638 bytesMESS:00:00:04.820505:0: brfs: File read: /mfs/sd/overlays/disable-bt.dtboMESS:00:00:04.861138:0: Loaded overlay 'disable-bt'MESS:00:00:04.863063:0: dtparam: i2c_arm=onMESS:00:00:04.972477:0: brfs: File read: 1073 bytesMESS:00:00:04.981860:0: brfs: File read: /mfs/sd/overlays/i2c-rtc.dtboMESS:00:00:05.058535:0: Loaded overlay 'i2c-rtc'MESS:00:00:05.060078:0: dtparam: pcf85063a=trueMESS:00:00:05.110530:0: brfs: File read: 7238 bytesMESS:00:00:05.116209:0: brfs: File read: /mfs/sd/overlays/dwc2.dtboMESS:00:00:05.134730:0: Loaded overlay 'dwc2'MESS:00:00:05.136008:0: dtparam: dr_mode=hostMESS:00:00:05.165957:0: brfs: File read: 801 bytesMESS:00:00:05.176618:0: brfs: File read: /mfs/sd/overlays/pcie-32bit-dma.dtboMESS:00:00:05.197118:0: Loaded overlay 'pcie-32bit-dma'MESS:00:00:05.224144:0: brfs: File read: 438 bytesMESS:00:00:05.240641:0: brfs: File read: /mfs/sd/overlays/vc4-kms-v3d.dtboMESS:00:00:05.257235:0: dterror: can't find symbol 'i2c2'MESS:00:00:05.259555:0: Failed to resolve overlay 'vc4-kms-v3d'MESS:00:00:05.265256:0: brfs: File read: 2703 bytesMESS:00:00:05.273641:0: brfs: File read: /mfs/sd/overlays/disable-bt.dtboMESS:00:00:05.314498:0: Loaded overlay 'disable-bt'MESS:00:00:05.316314:0: dtparam: ant2=trueMESS:00:00:05.503649:0: brfs: File read: 1073 bytesMESS:00:00:05.506125:0: brfs: File read: /mfs/sd/cmdline.txtMESS:00:00:05.510877:0: Read command line from file 'cmdline.txt':MESS:00:00:05.516740:0: 'pci=nomsi console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=squashfs,ext4 rootwait'MESS:00:00:05.768257:0: brfs: File read: 100 bytesMESS:00:00:07.900937:0: brfs: File read: /mfs/sd/kernel8.imgMESS:00:00:07.903541:0: Loading 'kernel8.img' to 0x80000 size 0xb35008MESS:00:00:07.923993:0: Kernel relocated to 0x200000MESS:00:00:07.925886:0: Device tree loaded to 0x2eff2a00 (size 0xd52b)MESS:00:00:07.935655:0: uart: Set PL011 baud rate to 103448.300000 HzMESS:00:00:07.941981:0: uart: Baud rate change done...MESS:00:00:07.944061:0: uart: Baud rate change done...MESS:00:00:07.949630:0: gpioman: gpioman_get_pin_num: pin SDCARD_CONTROL_POWER not defined[ 0.000000] Booting Linux on physical CPU 0x0000000000 [0x410fd083]...Code:
Dec 1 2021 15:02:25 Copyright (c) 2012 Broadcomversion 71bd3109023a0c8575585ba87cbb374d2eeb038f (clean) (release) (start_db)Code:
2024/07/30 14:02:35version 5f18ffb61682740ed1751ba59952843d7521314f (release)timestamp 1722344555update-time 1726693078capabilities 0x0000007fCode:
[all]BOOT_UART=1WAKE_ON_GPIO=1POWER_OFF_ON_HALT=0BOOT_ORDER=0xf25641ENABLE_SELF_UPDATE=1Code:
Linux VCS_C7C226 5.15.137 #0 SMP Tue Nov 14 13:38:11 2023 aarch64 GNU/LinuxCode:
# OpenWrt configinclude distroconfig.txt[all]# Place your custom settings here.dtparam=i2c_arm=ondtoverlay=i2c-rtc,pcf85063adtoverlay=dwc2,dr_mode=hostdtoverlay=pcie-32bit-dmadtoverlay=vc4-kms-v3d,cma-128dtoverlay=disable-btdtparam=ant2gpu_mem=128boot_delay=5uart_2ndstage=1enable_uart=1Code:
pci=nomsi console=serial0,115200 console=tty1 root=/dev/mmcblk0p2 rootfstype=squashfs,ext4 rootwaitCode:
00:00.0 PCI bridge: Broadcom Inc. and subsidiaries BCM2711 PCIe Bridge (rev 20) (prog-if 00 [Normal decode]) Device tree node: /sys/firmware/devicetree/base/scb/pcie@7d500000/pci@0,0 Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 45 Bus: primary=00, secondary=01, subordinate=01, sec-latency=0 Memory behind bridge: fff00000-000fffff [disabled] [32-bit] Prefetchable memory behind bridge: 600000000-6001fffff [size=2M] [32-bit] Secondary status: 66MHz- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- <SERR- <PERR- BridgeCtl: Parity- SERR+ NoISA- VGA- VGA16- MAbort- >Reset- FastB2B- PriDiscTmr- SecDiscTmr- DiscTmrStat- DiscTmrSERREn- Capabilities: [48] Power Management version 3 Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0+,D1-,D2-,D3hot+,D3cold-) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=1 PME- Capabilities: [ac] Express (v2) Root Port (Slot-), MSI 00 DevCap: MaxPayload 512 bytes, PhantFunc 0 ExtTag- RBE+ DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- RlxdOrd+ ExtTag- PhantFunc- AuxPwr+ NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr+ NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- LnkCap: Port #0, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <2us, L1 <4us ClockPM+ Surprise- LLActRep- BwNot+ ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s, Width x1 TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt+ RootCap: CRSVisible+ RootCtl: ErrCorrectable- ErrNon-Fatal- ErrFatal- PMEIntEna+ CRSVisible+ RootSta: PME ReqID 0000, PMEStatus- PMEPending- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ 10BitTagComp- 10BitTagReq- OBFF Via WAKE#, ExtFmt- EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- LN System CLS Not Supported, TPHComp- ExtTPHComp- ARIFwd- AtomicOpsCap: Routing- 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled, ARIFwd- AtomicOpsCtl: ReqEn- EgressBlck- LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS+ LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot LnkSta2: Current De-emphasis Level: -6dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported, DRS- DownstreamComp: Link Down - Not Determined Capabilities: [100 v1] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF+ MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP+ BadDLLP+ Rollover- Timeout- AdvNonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 RootCmd: CERptEn- NFERptEn- FERptEn- RootSta: CERcvd- MultCERcvd- UERcvd- MultUERcvd- FirstFatal- NonFatalMsg- FatalMsg- IntMsg 0 ErrorSrc: ERR_COR: 0000 ERR_FATAL/NONFATAL: 0000 Capabilities: [180 v1] Vendor Specific Information: ID=0000 Rev=0 Len=028 <?> Capabilities: [240 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+ PortCommonModeRestoreTime=8us PortTPowerOnTime=10us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- T_CommonMode=1us LTR1.2_Threshold=0ns L1SubCtl2: T_PwrOn=10us Kernel driver in use: pcieportlspci: Unable to load libkmod resources: error -201:00.0 Unclassified device [0002]: MEDIATEK Corp. MT7915E 802.11ax PCI Express Wireless Network Adapter (prog-if 80) Subsystem: MEDIATEK Corp. MT7915E 802.11ax PCI Express Wireless Network Adapter Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx- Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx- Latency: 0 Interrupt: pin A routed to IRQ 45 Region 0: Memory at 600000000 (64-bit, prefetchable) [size=1M] Region 2: Memory at 600100000 (64-bit, prefetchable) [size=16K] Region 4: Memory at 600104000 (64-bit, prefetchable) [size=4K] Capabilities: [80] Express (v2) Endpoint, MSI 00 DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <1us, L1 <4us ExtTag+ AttnBtn- AttnInd- PwrInd- RBE+ FLReset- SlotPowerLimit 0W DevCtl: CorrErr- NonFatalErr- FatalErr- UnsupReq- RlxdOrd+ ExtTag+ PhantFunc- AuxPwr- NoSnoop+ MaxPayload 128 bytes, MaxReadReq 512 bytes DevSta: CorrErr- NonFatalErr- FatalErr- UnsupReq- AuxPwr- TransPend- LnkCap: Port #1, Speed 5GT/s, Width x1, ASPM L0s L1, Exit Latency L0s <2us, L1 <8us ClockPM+ Surprise- LLActRep- BwNot- ASPMOptComp+ LnkCtl: ASPM Disabled; RCB 64 bytes, Disabled- CommClk- ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt- LnkSta: Speed 5GT/s, Width x1 TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt- DevCap2: Completion Timeout: Range ABCD, TimeoutDis+ NROPrPrP- LTR+ 10BitTagComp- 10BitTagReq- OBFF Not Supported, ExtFmt+ EETLPPrefix- EmergencyPowerReduction Not Supported, EmergencyPowerReductionInit- FRS- TPHComp- ExtTPHComp- AtomicOpsCap: 32bit- 64bit- 128bitCAS- DevCtl2: Completion Timeout: 50us to 50ms, TimeoutDis- LTR- 10BitTagReq- OBFF Disabled, AtomicOpsCtl: ReqEn- LnkCap2: Supported Link Speeds: 2.5-5GT/s, Crosslink- Retimer- 2Retimers- DRS- LnkCtl2: Target Link Speed: 5GT/s, EnterCompliance- SpeedDis- Transmit Margin: Normal Operating Range, EnterModifiedCompliance- ComplianceSOS- Compliance Preset/De-emphasis: -6dB de-emphasis, 0dB preshoot LnkSta2: Current De-emphasis Level: -3.5dB, EqualizationComplete- EqualizationPhase1- EqualizationPhase2- EqualizationPhase3- LinkEqualizationRequest- Retimer- 2Retimers- CrosslinkRes: unsupported Capabilities: [e0] MSI: Enable- Count=1/32 Maskable+ 64bit+ Address: 0000000000000000 Data: 0000 Masking: 00000000 Pending: 00000000 Capabilities: [f8] Power Management version 3 Flags: PMEClk- DSI+ D1+ D2+ AuxCurrent=0mA PME(D0+,D1+,D2+,D3hot+,D3cold+) Status: D0 NoSoftRst+ PME-Enable- DSel=0 DScale=0 PME- Capabilities: [100 v1] Vendor Specific Information: ID=1556 Rev=1 Len=008 <?> Capabilities: [108 v1] Latency Tolerance Reporting Max snoop latency: 0ns Max no snoop latency: 0ns Capabilities: [110 v1] L1 PM Substates L1SubCap: PCI-PM_L1.2+ PCI-PM_L1.1+ ASPM_L1.2+ ASPM_L1.1+ L1_PM_Substates+ PortCommonModeRestoreTime=3us PortTPowerOnTime=28us L1SubCtl1: PCI-PM_L1.2- PCI-PM_L1.1- ASPM_L1.2- ASPM_L1.1- T_CommonMode=0us LTR1.2_Threshold=0ns L1SubCtl2: T_PwrOn=10us Capabilities: [200 v2] Advanced Error Reporting UESta: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UEMsk: DLP- SDES- TLP- FCP- CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP- ECRC- UnsupReq- ACSViol- UESvrt: DLP+ SDES+ TLP- FCP+ CmpltTO- CmpltAbrt- UnxCmplt- RxOF- MalfTLP+ ECRC- UnsupReq- ACSViol- CESta: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr- CEMsk: RxErr- BadTLP- BadDLLP- Rollover- Timeout- AdvNonFatalErr+ AERCap: First Error Pointer: 00, ECRCGenCap- ECRCGenEn- ECRCChkCap- ECRCChkEn- MultHdrRecCap- MultHdrRecEn- TLPPfxPres- HdrLogCap- HeaderLog: 00000000 00000000 00000000 00000000 Kernel driver in use: mt7915eStatistics: Posted by subrosian — Tue Sep 24, 2024 8:15 pm — Replies 0 — Views 38