My long-term goal - to stream IMX296 frames over ROS2
The current roadblock: Ubuntu 22.04 with the IMX296, seems to be a libcamera/OS interaction issue
Hardware: CM4. Both ports have been verified to work correctly with other cameras (IMX477) and both cameras have been verified to work with another board running the rpi OS. Libcamera detects the imager, but has a frontend timeout. I don't believe that it's hardware, because the working board running RPI OS had the exact same error message before I reflashed it.
Board 1 (camera verification board):
I installed the same version of libcamera from source on my board running 22.04 (note that I already increased the frontend timeout to 5 seconds)
The same behavior (frontend timeouts with IMX296, works with IMX477) was observed on the board which is now running the rpi OS while it was running Ubuntu 22.04, and now that it has the rpi OS installed, I've also verified it works with two cameras.
I am wondering if I need to install the RP1 Camera Front End driver to Ubuntu somehow (modinfo rp1-cfe shows nothing)? But that does not explain to me why the IMX477 worked fine on the Ubuntu OS.
I'm still learning the Linux driver interactions, so any tips about learning more about that driver (I'm not actually finding much information on it) would be extremely helpful.
If it's not possible to get this to work in 22.04, I'm at least trying to understand why - is it that driver? Should this be a bug report on the Ubuntu repository?
Thank you in advance for any guidance!
The current roadblock: Ubuntu 22.04 with the IMX296, seems to be a libcamera/OS interaction issue
Hardware: CM4. Both ports have been verified to work correctly with other cameras (IMX477) and both cameras have been verified to work with another board running the rpi OS. Libcamera detects the imager, but has a frontend timeout. I don't believe that it's hardware, because the working board running RPI OS had the exact same error message before I reflashed it.
Board 1 (camera verification board):
Code:
$ uname -aLinux <HOSTNAME> 6.1.0-rpi7-rpi-v8 #1 SMP PREEMPT Debian 1:6.1.63-1+rpt1 (2023-11-24) aarch64 GNU/Linux$ rpicam-hello --versionrpicam-apps build: f74361ee6a56 23-11-2023 (16:52:48)libcamera build: v0.1.0+118-563cd78e$ rpicam-raw --list-cameras --verboseAvailable cameras-----------------0 : imx296 [1456x1088 10-bit RGGB] (/base/soc/i2c0mux/i2c@1/imx296@1a) Modes: 'SRGGB10_CSI2P' : 1456x1088 [60.38 fps - (0, 0)/1456x1088 crop] Available controls for 1456x1088 SRGGB10_CSI2P mode: ---------------------------------------------------- AeConstraintMode : [0..3] AeEnable : [false..true] AeExposureMode : [0..3] AeFlickerMode : [0..1] AeFlickerPeriod : [100..1000000] AeMeteringMode : [0..3] AnalogueGain : [1.000000..251.188644] AwbEnable : [false..true] AwbMode : [0..7] Brightness : [-1.000000..1.000000] ColourGains : [0.000000..32.000000] Contrast : [0.000000..32.000000] ExposureTime : [29..0] ExposureValue : [-8.000000..8.000000] FrameDurationLimits : [16562..15534444] HdrMode : [0..4] NoiseReductionMode : [0..4] Saturation : [0.000000..32.000000] ScalerCrop : [(0, 0)/64x64..(0, 0)/1456x1088] Sharpness : [0.000000..16.000000]$ rpicam-raw -o test0.raw --camera 0[0:54:41.636130515] [873] INFO Camera camera_manager.cpp:284 libcamera v0.1.0+118-563cd78e[0:54:41.680531332] [876] WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise[0:54:41.682851869] [876] INFO RPI vc4.cpp:444 Registered camera /base/soc/i2c0mux/i2c@1/imx296@1a to Unicam device /dev/media2 and ISP device /dev/media4[0:54:41.682980757] [876] INFO RPI pipeline_base.cpp:1142 Using configuration file '/usr/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'Mode selection for 640:480:12:P SRGGB10_CSI2P,1456x1088/0 - Score: 1357.84[0:54:41.684197562] [873] INFO Camera camera.cpp:1183 configuring streams: (0) 640x480-YUV420 (1) 1456x1088-SBGGR10_CSI2P[0:54:41.684643392] [876] INFO RPI vc4.cpp:608 Sensor: /base/soc/i2c0mux/i2c@1/imx296@1a - Selected sensor format: 1456x1088-SBGGR10_1X10 - Selected unicam format: 1456x1088-pBAARaw stream: 1456x1088 stride 1824 format SBGGR10_CSI2PI'm starting to go in debugging circles, so I'm hoping to get some advice to know where my understanding is breaking down.$ uname -a
Linux <HOSTNAME> 5.15.0-1047-raspi #50-Ubuntu SMP PREEMPT Fri Feb 9 13:48:00 UTC 2024 aarch64 aarch64 aarch64 GNU/Linux
$ rpicam-hello --version
rpicam-apps build: f74361ee6a56-intree 04-03-2024 (20:07:33)
libcamera build: v0.1.0+118-563cd78e
$ rpicam-raw --list-cameras --verbose
Available cameras
-----------------
0 : imx477 [4056x3040 12-bit RGGB] (/base/soc/i2c0mux/i2c@1/imx477@1a)
Modes: 'SRGGB10_CSI2P' : 1332x990 [120.05 fps - (696, 528)/2664x1980 crop]
'SRGGB12_CSI2P' : 2028x1080 [50.03 fps - (0, 440)/4056x2160 crop]
2028x1520 [40.01 fps - (0, 0)/4056x3040 crop]
4056x3040 [10.00 fps - (0, 0)/4056x3040 crop]
Available controls for 4056x3040 SRGGB12_CSI2P mode:
----------------------------------------------------
AeConstraintMode : [0..3]
AeEnable : [false..true]
AeExposureMode : [0..3]
AeFlickerMode : [0..1]
AeFlickerPeriod : [100..1000000]
AeMeteringMode : [0..3]
AnalogueGain : [1.000000..22.260870]
AwbEnable : [false..true]
AwbMode : [0..7]
Brightness : [-1.000000..1.000000]
ColourGains : [0.000000..32.000000]
Contrast : [0.000000..32.000000]
ExposureTime : [571..127156999]
ExposureValue : [-8.000000..8.000000]
FrameDurationLimits : [100000..239542857]
HdrMode : [0..4]
NoiseReductionMode : [0..4]
Saturation : [0.000000..32.000000]
ScalerCrop : [(0, 0)/64x64..(0, 0)/4056x3040]
Sharpness : [0.000000..16.000000]
1 : imx296 [1456x1088 10-bit RGGB] (/base/soc/i2c0mux/i2c@0/imx296@1a)
Modes: 'SRGGB10_CSI2P' : 1456x1088 [60.38 fps - (0, 0)/1456x1088 crop]
Available controls for 1456x1088 SRGGB10_CSI2P mode:
----------------------------------------------------
AeConstraintMode : [0..3]
AeEnable : [false..true]
AeExposureMode : [0..3]
AeFlickerMode : [0..1]
AeFlickerPeriod : [100..1000000]
AeMeteringMode : [0..3]
AnalogueGain : [1.000000..251.188644]
AwbEnable : [false..true]
AwbMode : [0..7]
Brightness : [-1.000000..1.000000]
ColourGains : [0.000000..32.000000]
Contrast : [0.000000..32.000000]
ExposureTime : [29..0]
ExposureValue : [-8.000000..8.000000]
FrameDurationLimits : [16562..15534444]
HdrMode : [0..4]
NoiseReductionMode : [0..4]
Saturation : [0.000000..32.000000]
ScalerCrop : [(0, 0)/64x64..(0, 0)/1456x1088]
Sharpness : [0.000000..16.000000]
$ rpicam-raw -o test0.raw --camera 0
[0:13:51.801444759] [1445] INFO Camera camera_manager.cpp:284 libcamera v0.1.0+118-563cd78e
[0:13:51.857328897] [1448] WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[0:13:51.860540887] [1448] INFO RPI vc4.cpp:444 Registered camera /base/soc/i2c0mux/i2c@0/imx296@1a to Unicam device /dev/media0 and ISP device /dev/media2
[0:13:51.860646109] [1448] INFO RPI pipeline_base.cpp:1142 Using configuration file '/usr/local/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
[0:13:51.874277008] [1448] WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[0:13:51.877752755] [1448] INFO RPI vc4.cpp:444 Registered camera /base/soc/i2c0mux/i2c@1/imx477@1a to Unicam device /dev/media1 and ISP device /dev/media4
[0:13:51.877854939] [1448] INFO RPI pipeline_base.cpp:1142 Using configuration file '/usr/local/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
Mode selection for 640:480:12:P
SRGGB10_CSI2P,1332x990/0 - Score: 1305.05
SRGGB12_CSI2P,2028x1080/0 - Score: 701.167
SRGGB12_CSI2P,2028x1520/0 - Score: 607.329
SRGGB12_CSI2P,4056x3040/0 - Score: 1494.33
[0:13:51.879616035] [1445] INFO Camera camera.cpp:1183 configuring streams: (0) 640x480-YUV420 (1) 2028x1520-SBGGR12_CSI2P
[0:13:51.880288677] [1448] INFO RPI vc4.cpp:608 Sensor: /base/soc/i2c0mux/i2c@1/imx477@1a - Selected sensor format: 2028x1520-SBGGR12_1X12 - Selected unicam format: 2028x1520-pBCC
Raw stream: 2028x1520 stride 3072 format SBGGR12_CSI2P
$ rpicam-raw -o test1.raw --camera 1
[0:13:33.161022761] [1431] INFO Camera camera_manager.cpp:284 libcamera v0.1.0+118-563cd78e
[0:13:33.218136499] [1434] WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[0:13:33.221390119] [1434] INFO RPI vc4.cpp:444 Registered camera /base/soc/i2c0mux/i2c@0/imx296@1a to Unicam device /dev/media0 and ISP device /dev/media2
[0:13:33.221498451] [1434] INFO RPI pipeline_base.cpp:1142 Using configuration file '/usr/local/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
[0:13:33.234553040] [1434] WARN RPiSdn sdn.cpp:39 Using legacy SDN tuning - please consider moving SDN inside rpi.denoise
[0:13:33.238026157] [1434] INFO RPI vc4.cpp:444 Registered camera /base/soc/i2c0mux/i2c@1/imx477@1a to Unicam device /dev/media1 and ISP device /dev/media4
[0:13:33.238129323] [1434] INFO RPI pipeline_base.cpp:1142 Using configuration file '/usr/local/share/libcamera/pipeline/rpi/vc4/rpi_apps.yaml'
Mode selection for 640:480:12:P
SRGGB10_CSI2P,1456x1088/0 - Score: 1357.84
[0:13:33.239571440] [1431] INFO Camera camera.cpp:1183 configuring streams: (0) 640x480-YUV420 (1) 1456x1088-SBGGR10_CSI2P
[0:13:33.240205360] [1434] INFO RPI vc4.cpp:608 Sensor: /base/soc/i2c0mux/i2c@0/imx296@1a - Selected sensor format: 1456x1088-SBGGR10_1X10 - Selected unicam format: 1456x1088-pBAA
[0:13:33.770167806] [1434] INFO V4L2 v4l2_videodevice.cpp:1820 /dev/video0[20:cap]: Zero sequence expected for first frame (got 5)
[0:13:38.874166884] [1434] WARN V4L2 v4l2_videodevice.cpp:2007 /dev/video0[20:cap]: Dequeue timer of 5000000.00us has expired!
[0:13:38.874321697] [1434] ERROR RPI pipeline_base.cpp:1374 Camera frontend has timed out!
[0:13:38.874384160] [1434] ERROR RPI pipeline_base.cpp:1375 Please check that your camera sensor connector is attached securely.
[0:13:38.874444863] [1434] ERROR RPI pipeline_base.cpp:1376 Alternatively, try another cable and/or sensor.
ERROR: Device timeout detected, attempting a restart!!!
^C
The same behavior (frontend timeouts with IMX296, works with IMX477) was observed on the board which is now running the rpi OS while it was running Ubuntu 22.04, and now that it has the rpi OS installed, I've also verified it works with two cameras.
I am wondering if I need to install the RP1 Camera Front End driver to Ubuntu somehow (modinfo rp1-cfe shows nothing)? But that does not explain to me why the IMX477 worked fine on the Ubuntu OS.
I'm still learning the Linux driver interactions, so any tips about learning more about that driver (I'm not actually finding much information on it) would be extremely helpful.
If it's not possible to get this to work in 22.04, I'm at least trying to understand why - is it that driver? Should this be a bug report on the Ubuntu repository?
Thank you in advance for any guidance!
Statistics: Posted by nb-5 — Mon Mar 04, 2024 9:14 pm — Replies 0 — Views 26