Call us in Seattle
+1 (800) 871-6550
Blog:
NXP i.MX 6、メインラインカーネルでカメラとハードウェアアクセラレーテッド（VPU）におけるHMIを利用
Thursday, March 21, 2024
動機
ベンダーが対応しているダウンストリームのカーネルからメインラインのカーネルへ移行するのは、困難な作業となる場合があります。ベンダーのサポート（通常クローズドソースのバイナリによるもの）と引き換えに、完全にオープンなツールとソースコードを利用することになります。
NXP i.MX6のケースでは、VPUやGPUといった一部のクローズドソースIPは、すでに長期に渡ってオープンにサポートされてきました。このブログでは、メインカーネルでのサポートに関する変更について理解しやすいよう、詳細を説明します。
今回は、NXP i.MX6搭載のApalis iMX6とLinuxカーネルv6.1を使用します。メインラインのカーネルの利点の1つは、いったんサブシステム含んで保守を行ってしまえば、カーネルの更新時にも同じような機能となることが予想可能なことで、最低限の保守努力で最新のセキュリティーパッチを利用できるメリットもあります。
dmesg
root@apalis-imx6-10870254:~# dmesg [ 0.000000] Booting Linux on physical CPU 0x0 [ 0.000000] Linux version 6.1.71-6.5.0+git.38fb82ecd144 (oe-user@oe-host) (arm-tdx-linux-gnueabi-gcc (GCC) 11.4.0, GNU ld (GNU Binutils) 2.38.20220708) #1 SMP Fri Jan 5 14:18:41 UTC 2024 [ 0.000000] CPU: ARMv7 Processor [412fc09a] revision 10 (ARMv7), cr=10c5387d [ 0.000000] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing instruction cache [ 0.000000] OF: fdt: Machine model: Toradex Apalis iMX6Q/D Module on Apalis Evaluation Board [ 0.000000] Memory policy: Data cache writealloc [ 0.000000] cma: Reserved 256 MiB at 0x40000000 [ 0.000000] Zone ranges: [ 0.000000] Normal [mem 0x0000000010000000-0x0000000035ffffff] [ 0.000000] HighMem [mem 0x0000000036000000-0x000000004fffffff] [ 0.000000] Movable zone start for each node [ 0.000000] Early memory node ranges [ 0.000000] node 0: [mem 0x0000000010000000-0x000000004fffffff] [ 0.000000] Initmem setup node 0 [mem 0x0000000010000000-0x000000004fffffff] [ 0.000000] percpu: Embedded 12 pages/cpu s18836 r8192 d22124 u49152 [ 0.000000] pcpu-alloc: s18836 r8192 d22124 u49152 alloc=12*4096 [ 0.000000] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 [ 0.000000] Built 1 zonelists, mobility grouping on. Total pages: 260928 [ 0.000000] Kernel command line: enable_wait_mode=off vmalloc=400M root=PARTUUID=5b73db23-02 ro rootwait fec_mac=00:14:2d:a5:dd:ee consoleblank=0 no_console_suspend=1 console=tty1 console=ttymxc0,115200n8 mxc_hdmi.only_cea=1 fbmem=32M [ 0.000000] Unknown kernel command line parameters "enable_wait_mode=off fec_mac=00:14:2d:a5:dd:ee fbmem=32M", will be passed to user space. [ 0.000000] Dentry cache hash table entries: 131072 (order: 7, 524288 bytes, linear) [ 0.000000] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes, linear) [ 0.000000] mem auto-init: stack:off, heap alloc:off, heap free:off [ 0.000000] Memory: 757924K/1048576K available (11264K kernel code, 1237K rwdata, 3900K rodata, 1024K init, 437K bss, 28508K reserved, 262144K cma-reserved, 163840K highmem) [ 0.000000] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1 [ 0.000000] rcu: Hierarchical RCU implementation. [ 0.000000] rcu: RCU event tracing is enabled. [ 0.000000] Tracing variant of Tasks RCU enabled. [ 0.000000] rcu: RCU calculated value of scheduler-enlistment delay is 10 jiffies. [ 0.000000] NR_IRQS: 16, nr_irqs: 16, preallocated irqs: 16 [ 0.000000] L2C-310 errata 752271 769419 enabled [ 0.000000] L2C-310 enabling early BRESP for Cortex-A9 [ 0.000000] L2C-310 full line of zeros enabled for Cortex-A9 [ 0.000000] L2C-310 ID prefetch enabled, offset 16 lines [ 0.000000] L2C-310 dynamic clock gating enabled, standby mode enabled [ 0.000000] L2C-310 cache controller enabled, 16 ways, 1024 kB [ 0.000000] L2C-310: CACHE_ID 0x410000c7, AUX_CTRL 0x76470001 [ 0.000000] rcu: srcu_init: Setting srcu_struct sizes based on contention. [ 0.000000] Switching to timer-based delay loop, resolution 333ns [ 0.000001] sched_clock: 32 bits at 3000kHz, resolution 333ns, wraps every 715827882841ns [ 0.000018] clocksource: mxc_timer1: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 637086815595 ns [ 0.001519] Console: colour dummy device 80x30 [ 0.001961] printk: console [tty1] enabled [ 0.002012] Calibrating delay loop (skipped), value calculated using timer frequency.. 6.00 BogoMIPS (lpj=30000) [ 0.002046] CPU: Testing write buffer coherency: ok [ 0.002094] CPU0: Spectre v2: using BPIALL workaround [ 0.002115] pid_max: default: 32768 minimum: 301 [ 0.002306] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes, linear) [ 0.002348] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 bytes, linear) [ 0.003269] CPU0: thread -1, cpu 0, socket 0, mpidr 80000000 [ 0.004306] cblist_init_generic: Setting adjustable number of callback queues. [ 0.004338] cblist_init_generic: Setting shift to 2 and lim to 1. [ 0.004521] Setting up static identity map for 0x10100000 - 0x10100078 [ 0.004721] rcu: Hierarchical SRCU implementation. [ 0.004744] rcu: Max phase no-delay instances is 1000. [ 0.006240] smp: Bringing up secondary CPUs ... [ 0.007240] CPU1: thread -1, cpu 1, socket 0, mpidr 80000001 [ 0.007257] CPU1: Spectre v2: using BPIALL workaround [ 0.008338] CPU2: thread -1, cpu 2, socket 0, mpidr 80000002 [ 0.008354] CPU2: Spectre v2: using BPIALL workaround [ 0.009417] CPU3: thread -1, cpu 3, socket 0, mpidr 80000003 [ 0.009434] CPU3: Spectre v2: using BPIALL workaround [ 0.009568] smp: Brought up 1 node, 4 CPUs [ 0.009598] SMP: Total of 4 processors activated (24.00 BogoMIPS). [ 0.009621] CPU: All CPU(s) started in SVC mode. [ 0.010237] devtmpfs: initialized [ 0.021729] VFP support v0.3: implementor 41 architecture 3 part 30 variant 9 rev 4 [ 0.022004] clocksource: jiffies: mask: 0xffffffff max_cycles: 0xffffffff, max_idle_ns: 19112604462750000 ns [ 0.022051] futex hash table entries: 1024 (order: 4, 65536 bytes, linear) [ 0.039089] pinctrl core: initialized pinctrl subsystem [ 0.040891] NET: Registered PF_NETLINK/PF_ROUTE protocol family [ 0.048428] DMA: preallocated 256 KiB pool for atomic coherent allocations [ 0.049493] thermal_sys: Registered thermal governor 'step_wise' [ 0.049557] cpuidle: using governor menu [ 0.049768] CPU identified as i.MX6Q, silicon rev 1.6 [ 0.058236] platform soc: Fixed dependency cycle(s) with /soc/bus@2000000/gpc@20dc000 [ 0.070532] platform 20e0000.pinctrl: Fixed dependency cycle(s) with /soc/bus@2000000/pinctrl@20e0000/apalisgpio8grp [ 0.070590] platform 20e0000.pinctrl: Fixed dependency cycle(s) with /soc/bus@2000000/pinctrl@20e0000/apalisgpio7grp [ 0.070634] platform 20e0000.pinctrl: Fixed dependency cycle(s) with /soc/bus@2000000/pinctrl@20e0000/apalisgpio6grp [ 0.070674] platform 20e0000.pinctrl: Fixed dependency cycle(s) with /soc/bus@2000000/pinctrl@20e0000/apalisgpio5grp [ 0.070714] platform 20e0000.pinctrl: Fixed dependency cycle(s) with /soc/bus@2000000/pinctrl@20e0000/apalisgpio4grp [ 0.070751] platform 20e0000.pinctrl: Fixed dependency cycle(s) with /soc/bus@2000000/pinctrl@20e0000/apalisgpio3grp [ 0.070789] platform 20e0000.pinctrl: Fixed dependency cycle(s) with /soc/bus@2000000/pinctrl@20e0000/apalisgpio2grp [ 0.070826] platform 20e0000.pinctrl: Fixed dependency cycle(s) with /soc/bus@2000000/pinctrl@20e0000/apalisgpio1grp [ 0.075724] platform 2400000.ipu: Fixed dependency cycle(s) with /soc/hdmi@120000/ports/port@1/endpoint [ 0.075794] platform 2400000.ipu: Fixed dependency cycle(s) with /soc/hdmi@120000/ports/port@0/endpoint [ 0.076695] platform 2800000.ipu: Fixed dependency cycle(s) with /soc/hdmi@120000/ports/port@3/endpoint [ 0.076768] platform 2800000.ipu: Fixed dependency cycle(s) with /soc/hdmi@120000/ports/port@2/endpoint [ 0.076817] platform 2800000.ipu: Fixed dependency cycle(s) with /soc/bus@2000000/iomuxc-gpr@20e0000/ipu2_csi1_mux/port@2/endpoint [ 0.081738] hw-breakpoint: found 5 (+1 reserved) breakpoint and 1 watchpoint registers. [ 0.081774] hw-breakpoint: maximum watchpoint size is 4 bytes. [ 0.082906] imx6q-pinctrl 20e0000.pinctrl: initialized IMX pinctrl driver [ 0.095776] kprobes: kprobe jump-optimization is enabled. All kprobes are optimized if possible. [ 0.112322] SCSI subsystem initialized [ 0.112400] libata version 3.00 loaded. [ 0.112634] usbcore: registered new interface driver usbfs [ 0.112701] usbcore: registered new interface driver hub [ 0.112761] usbcore: registered new device driver usb [ 0.112901] usb_phy_generic usbphynop1: supply vcc not found, using dummy regulator [ 0.113074] usb_phy_generic usbphynop1: dummy supplies not allowed for exclusive requests [ 0.113246] usb_phy_generic usbphynop2: supply vcc not found, using dummy regulator [ 0.113429] usb_phy_generic usbphynop2: dummy supplies not allowed for exclusive requests [ 0.115197] i2c i2c-0: IMX I2C adapter registered [ 0.120371] stmpe-i2c 2-0041: stmpe811 detected, chip id: 0x811 [ 0.210599] i2c i2c-2: IMX I2C adapter registered [ 0.211579] i2c i2c-1: IMX I2C adapter registered [ 0.211877] mc: Linux media interface: v0.10 [ 0.211956] videodev: Linux video capture interface: v2.00 [ 0.212059] pps_core: LinuxPPS API ver. 1 registered [ 0.212079] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it> [ 0.212121] PTP clock support registered [ 0.212659] Advanced Linux Sound Architecture Driver Initialized. [ 0.213760] Bluetooth: Core ver 2.22 [ 0.213821] NET: Registered PF_BLUETOOTH protocol family [ 0.213841] Bluetooth: HCI device and connection manager initialized [ 0.213870] Bluetooth: HCI socket layer initialized [ 0.213894] Bluetooth: L2CAP socket layer initialized [ 0.213926] Bluetooth: SCO socket layer initialized [ 0.214586] clocksource: Switched to clocksource mxc_timer1 [ 0.214887] VFS: Disk quotas dquot_6.6.0 [ 0.214980] VFS: Dquot-cache hash table entries: 1024 (order 0, 4096 bytes) [ 0.225650] NET: Registered PF_INET protocol family [ 0.226102] IP idents hash table entries: 16384 (order: 5, 131072 bytes, linear) [ 0.228132] tcp_listen_portaddr_hash hash table entries: 512 (order: 0, 4096 bytes, linear) [ 0.228184] Table-perturb hash table entries: 65536 (order: 6, 262144 bytes, linear) [ 0.228220] TCP established hash table entries: 8192 (order: 3, 32768 bytes, linear) [ 0.228319] TCP bind hash table entries: 8192 (order: 5, 131072 bytes, linear) [ 0.228622] TCP: Hash tables configured (established 8192 bind 8192) [ 0.228849] UDP hash table entries: 512 (order: 2, 16384 bytes, linear) [ 0.228921] UDP-Lite hash table entries: 512 (order: 2, 16384 bytes, linear) [ 0.229155] NET: Registered PF_UNIX/PF_LOCAL protocol family [ 0.229717] RPC: Registered named UNIX socket transport module. [ 0.229746] RPC: Registered udp transport module. [ 0.229764] RPC: Registered tcp transport module. [ 0.229781] RPC: Registered tcp NFSv4.1 backchannel transport module. [ 0.229807] PCI: CLS 0 bytes, default 64 [ 0.230815] armv7-pmu pmu: hw perfevents: no interrupt-affinity property, guessing. [ 0.231072] hw perfevents: enabled with armv7_cortex_a9 PMU driver, 7 counters available [ 0.232902] Initialise system trusted keyrings [ 0.233164] workingset: timestamp_bits=30 max_order=18 bucket_order=0 [ 0.238416] NFS: Registering the id_resolver key type [ 0.238476] Key type id_resolver registered [ 0.238498] Key type id_legacy registered [ 0.238589] nfs4filelayout_init: NFSv4 File Layout Driver Registering... [ 0.238615] nfs4flexfilelayout_init: NFSv4 Flexfile Layout Driver Registering... [ 0.333769] Key type asymmetric registered [ 0.333798] Asymmetric key parser 'x509' registered [ 0.333982] bounce: pool size: 64 pages [ 0.334077] io scheduler mq-deadline registered [ 0.334099] io scheduler kyber registered [ 0.338651] imx6q-pcie 1ffc000.pcie: host bridge /soc/pcie@1ffc000 ranges: [ 0.338737] imx6q-pcie 1ffc000.pcie: IO 0x0001f80000..0x0001f8ffff -> 0x0000000000 [ 0.338788] imx6q-pcie 1ffc000.pcie: MEM 0x0001000000..0x0001efffff -> 0x0001000000 [ 0.340480] mxs-dma 110000.dma-apbh: initialized [ 0.345245] pfuze100-regulator 2-0008: Full layer: 2, Metal layer: 1 [ 0.345971] pfuze100-regulator 2-0008: FAB: 0, FIN: 0 [ 0.345997] pfuze100-regulator 2-0008: pfuze100 found. [ 0.363657] 2020000.serial: ttymxc0 at MMIO 0x2020000 (irq = 270, base_baud = 5000000) is a IMX [ 0.701797] imx6q-pcie 1ffc000.pcie: iATU unroll: disabled [ 0.707723] printk: console [ttymxc0] enabled [ 0.714253] imx6q-pcie 1ffc000.pcie: iATU regions: 4 ob, 4 ib, align 64K, limit 4G [ 0.720259] 21e8000.serial: ttymxc1 at MMIO 0x21e8000 (irq = 272, base_baud = 5000000) is a IMX [ 1.409208] 21f0000.serial: ttymxc3 at MMIO 0x21f0000 (irq = 273, base_baud = 5000000) is a IMX [ 1.418743] 21f4000.serial: ttymxc4 at MMIO 0x21f4000 (irq = 274, base_baud = 5000000) is a IMX [ 1.430141] dwhdmi-imx 120000.hdmi: Detected HDMI TX controller v1.30a with HDCP (DWC HDMI 3D TX PHY) [ 1.439897] dwhdmi-imx 120000.hdmi: registered DesignWare HDMI I2C bus driver [ 1.460449] etnaviv etnaviv: bound 130000.gpu (ops gpu_ops) [ 1.466341] etnaviv etnaviv: bound 134000.gpu (ops gpu_ops) [ 1.472197] etnaviv etnaviv: bound 2204000.gpu (ops gpu_ops) [ 1.477923] etnaviv-gpu 130000.gpu: model: GC2000, revision: 5108 [ 1.484225] etnaviv-gpu 134000.gpu: model: GC320, revision: 5007 [ 1.490348] etnaviv-gpu 2204000.gpu: model: GC355, revision: 1215 [ 1.496494] etnaviv-gpu 2204000.gpu: Ignoring GPU with VG and FE2.0 [ 1.503224] [drm] Initialized etnaviv 1.3.0 20151214 for etnaviv on minor 0 [ 1.512086] imx-ipuv3 2400000.ipu: IPUv3H probed [ 1.517857] Stack Depot allocating hash table of 65536 entries with kvcalloc [ 1.526265] imx-drm display-subsystem: bound imx-ipuv3-crtc.2 (ops ipu_crtc_ops) [ 1.533821] imx-drm display-subsystem: bound imx-ipuv3-crtc.3 (ops ipu_crtc_ops) [ 1.541425] imx-drm display-subsystem: bound imx-ipuv3-crtc.6 (ops ipu_crtc_ops) [ 1.549001] imx-drm display-subsystem: bound imx-ipuv3-crtc.7 (ops ipu_crtc_ops) [ 1.556510] imx-drm display-subsystem: bound 120000.hdmi (ops dw_hdmi_imx_ops) [ 1.564236] [drm] Initialized imx-drm 1.0.0 20120507 for display-subsystem on minor 1 [ 1.724520] imx6q-pcie 1ffc000.pcie: Phy link never came up [ 1.759682] Console: switching to colour frame buffer device 240x67 [ 1.787824] imx-drm display-subsystem: [drm] fb0: imx-drmdrmfb frame buffer device [ 1.795626] imx-ipuv3 2800000.ipu: IPUv3H probed [ 1.810402] brd: module loaded [ 1.821663] loop: module loaded [ 1.826047] ahci-imx 2200000.sata: fsl,transmit-level-mV not specified, using 00000024 [ 1.834086] ahci-imx 2200000.sata: fsl,transmit-boost-mdB not specified, using 00000480 [ 1.842225] ahci-imx 2200000.sata: fsl,transmit-atten-16ths not specified, using 00002000 [ 1.850526] ahci-imx 2200000.sata: fsl,receive-eq-mdB not specified, using 05000000 [ 1.858413] ahci-imx 2200000.sata: supply ahci not found, using dummy regulator [ 1.866035] ahci-imx 2200000.sata: supply phy not found, using dummy regulator [ 1.873459] ahci-imx 2200000.sata: supply target not found, using dummy regulator [ 1.883960] ahci-imx 2200000.sata: SSS flag set, parallel bus scan disabled [ 1.891079] ahci-imx 2200000.sata: AHCI 0001.0300 32 slots 1 ports 3 Gbps 0x1 impl platform mode [ 1.900013] ahci-imx 2200000.sata: flags: ncq sntf stag pm led clo only pmp pio slum part ccc apst [ 1.911035] scsi host0: ahci-imx [ 1.914727] ata1: SATA max UDMA/133 mmio [mem 0x02200000-0x02203fff] port 0x100 irq 287 [ 1.926765] CAN device driver interface [ 1.946350] pps pps0: new PPS source ptp0 [ 1.954980] fec 2188000.ethernet eth0: registered PHC device 0 [ 1.962025] usbcore: registered new interface driver usb-storage [ 1.974568] rtc-ds1307 0-0068: oscillator failed, set time! [ 1.980502] rtc-ds1307 0-0068: registered as rtc0 [ 1.986752] rtc-ds1307 0-0068: hctosys: unable to read the hardware clock [ 1.995066] snvs_rtc 20cc000.snvs:snvs-rtc-lp: registered as rtc1 [ 2.001437] i2c_dev: i2c /dev entries driver [ 2.008756] Bluetooth: HCI UART driver ver 2.3 [ 2.013286] Bluetooth: HCI UART protocol H4 registered [ 2.018576] Bluetooth: HCI UART protocol LL registered [ 2.024792] sdhci: Secure Digital Host Controller Interface driver [ 2.031072] sdhci: Copyright(c) Pierre Ossman [ 2.035518] sdhci-pltfm: SDHCI platform and OF driver helper [ 2.046923] sdhci-esdhc-imx 2190000.mmc: Got CD GPIO [ 2.047006] caam 2100000.crypto: Entropy delay = 3200 [ 2.047089] sdhci-esdhc-imx 2194000.mmc: Got CD GPIO [ 2.104864] mmc1: SDHCI controller on 2194000.mmc [2194000.mmc] using ADMA [ 2.133779] caam 2100000.crypto: Instantiated RNG4 SH0 [ 2.194541] caam 2100000.crypto: Instantiated RNG4 SH1 [ 2.203318] caam 2100000.crypto: device ID = 0x0a16010000000000 (Era 4) [ 2.213515] caam 2100000.crypto: job rings = 2, qi = 0 [ 2.225586] caam algorithms registered in /proc/crypto [ 2.234635] mmc0: SDHCI controller on 2190000.mmc [2190000.mmc] using ADMA [ 2.234779] caam 2100000.crypto: registering rng-caam [ 2.253643] mmc2: SDHCI controller on 2198000.mmc [2198000.mmc] using ADMA [ 2.256772] ata1: SATA link down (SStatus 0 SControl 300) [ 2.273513] ahci-imx 2200000.sata: no device found, disabling link. [ 2.283475] ahci-imx 2200000.sata: pass ahci_imx..hotplug=1 to enable hotplug [ 2.373723] mmc2: new DDR MMC card at address 0001 [ 2.376896] random: crng init done [ 2.378965] caam 2100000.crypto: rng crypto API alg registered prng-caam [ 2.379535] mmcblk2: mmc2:0001 S40004 3.64 GiB [ 2.407894] mmcblk2: p1 p2 [ 2.415228] mmcblk2boot0: mmc2:0001 S40004 4.00 MiB [ 2.425938] mmcblk2boot1: mmc2:0001 S40004 4.00 MiB [ 2.436212] mmcblk2rpmb: mmc2:0001 S40004 4.00 MiB, chardev (242:0) [ 2.447517] usbcore: registered new interface driver usbhid [ 2.456614] usbhid: USB HID core driver [ 2.464827] imx-ipuv3-csi imx-ipuv3-csi.0: Registered ipu1_csi0 capture as /dev/video0 [ 2.476479] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpenc capture as /dev/video1 [ 2.487808] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpvf capture as /dev/video2 [ 2.499175] imx-ipuv3-csi imx-ipuv3-csi.1: Registered ipu1_csi1 capture as /dev/video3 [ 2.510908] imx-ipuv3-csi imx-ipuv3-csi.4: Registered ipu2_csi0 capture as /dev/video4 [ 2.522422] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5 [ 2.533685] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6 [ 2.544953] imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7 [ 2.557564] stmpe-adc stmpe-adc: DMA mask not set [ 2.578113] sgtl5000 2-000a: sgtl5000 revision 0x11 [ 2.724628] imx6q-pcie 1ffc000.pcie: Phy link never came up [ 2.733704] imx6q-pcie 1ffc000.pcie: PCI host bridge to bus 0000:00 [ 2.743367] pci_bus 0000:00: root bus resource [bus 00-ff] [ 2.752236] pci_bus 0000:00: root bus resource [io 0x0000-0xffff] [ 2.761758] pci_bus 0000:00: root bus resource [mem 0x01000000-0x01efffff] [ 2.772010] pci 0000:00:00.0: [16c3:abcd] type 01 class 0x060400 [ 2.781363] pci 0000:00:00.0: reg 0x10: [mem 0x00000000-0x000fffff] [ 2.790985] pci 0000:00:00.0: reg 0x38: [mem 0x00000000-0x0000ffff pref] [ 2.801022] pci 0000:00:00.0: Limiting cfg_size to 512 [ 2.809454] pci 0000:00:00.0: supports D1 [ 2.816649] pci 0000:00:00.0: PME# supported from D0 D1 D3hot D3cold [ 2.831110] PCI: bus0: Fast back to back transfers disabled [ 2.840130] PCI: bus1: Fast back to back transfers enabled [ 2.848796] pci 0000:00:00.0: BAR 0: assigned [mem 0x01000000-0x010fffff] [ 2.858771] pci 0000:00:00.0: BAR 6: assigned [mem 0x01100000-0x0110ffff pref] [ 2.869155] pci 0000:00:00.0: PCI bridge to [bus 01-ff] [ 3.136324] fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW [ 3.149714] NET: Registered PF_INET6 protocol family [ 3.159092] Segment Routing with IPv6 [ 3.165941] In-situ OAM (IOAM) with IPv6 [ 3.173060] sit: IPv6, IPv4 and MPLS over IPv4 tunneling driver [ 3.182751] NET: Registered PF_PACKET protocol family [ 3.191000] can: controller area network core [ 3.198618] NET: Registered PF_CAN protocol family [ 3.206609] can: raw protocol [ 3.212718] can: broadcast manager protocol [ 3.220078] can: netlink gateway - max_hops=1 [ 3.227663] Key type dns_resolver registered [ 3.237237] Registering SWP/SWPB emulation handler [ 3.248766] Loading compiled-in X.509 certificates [ 3.303127] ci_hdrc ci_hdrc.1: EHCI Host Controller [ 3.311053] ci_hdrc ci_hdrc.1: new USB bus registered, assigned bus number 1 [ 3.354619] ci_hdrc ci_hdrc.1: USB 2.0 started, EHCI 1.00 [ 3.366725] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002, bcdDevice= 6.01 [ 3.377882] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1 [ 3.387712] usb usb1: Product: EHCI Host Controller [ 3.395137] usb usb1: Manufacturer: Linux 6.1.71-6.5.0+git.38fb82ecd144 ehci_hcd [ 3.405105] usb usb1: SerialNumber: ci_hdrc.1 [ 3.412510] hub 1-0:1.0: USB hub found [ 3.418838] hub 1-0:1.0: 1 port detected [ 3.427818] imx-media: Registered ipu_ic_pp csc/scaler as /dev/video8 [ 3.437697] imx_thermal 20c8000.anatop:tempmon: Extended Commercial CPU temperature grade - max:105C critical:105C passive:95C [ 3.453013] fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW [ 3.466216] fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW [ 3.478967] input: gpio-keys as /devices/platform/gpio-keys/input/input0 [ 3.490564] ALSA device list: [ 3.491118] fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW [ 3.493544] No soundcards found. [ 3.529660] EXT4-fs (mmcblk2p2): INFO: recovery required on readonly filesystem [ 3.540358] EXT4-fs (mmcblk2p2): write access will be enabled during recovery [ 3.584258] EXT4-fs (mmcblk2p2): recovery complete [ 3.594145] EXT4-fs (mmcblk2p2): mounted filesystem with ordered data mode. Quota mode: none. [ 3.606090] VFS: Mounted root (ext4 filesystem) readonly on device 179:2. [ 3.619624] devtmpfs: mounted [ 3.627476] Freeing unused kernel image (initmem) memory: 1024K [ 3.636472] Run /sbin/init as init process [ 3.643255] with arguments: [ 3.643264] /sbin/init [ 3.643270] with environment: [ 3.643276] HOME=/ [ 3.643281] TERM=linux [ 3.643286] enable_wait_mode=off [ 3.643292] fec_mac=00:14:2d:a5:dd:ee [ 3.643297] fbmem=32M [ 3.714627] usb 1-1: new high-speed USB device number 2 using ci_hdrc [ 3.815423] systemd[1]: System time before build time, advancing clock. [ 3.864849] systemd[1]: systemd 250.5+ running in system mode (+PAM -AUDIT -SELINUX -APPARMOR +IMA -SMACK +SECCOMP -GCRYPT -GNUTLS -OPENSSL +ACL +BLKID -CURL -ELFUTILS -FIDO2 -IDN2 -IDN -IPTC +KMOD -LIBCRYPTSETUP +LIBFDISK -PCRE2 -PWQUALITY -P11KIT -QRENCODE -BZIP2 -LZ4 -XZ -ZLIB +ZSTD -BPF_FRAMEWORK +XKBCOMMON +UTMP +SYSVINIT default-hierarchy=hybrid) [ 3.903109] systemd[1]: Detected architecture arm. [ 3.945304] usb 1-1: New USB device found, idVendor=0424, idProduct=2514, bcdDevice= b.b3 [ 3.958368] usb 1-1: New USB device strings: Mfr=0, Product=0, SerialNumber=0 [ 3.958671] systemd[1]: Hostname set to. [ 3.971696] hub 1-1:1.0: USB hub found [ 3.987041] hub 1-1:1.0: 4 ports detected [ 3.997930] fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW [ 4.011625] fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW [ 4.657471] systemd[1]: Queued start job for default target Graphical Interface. [ 4.746762] systemd[1]: Created slice Slice /system/getty. [ 4.786426] systemd[1]: Created slice Slice /system/modprobe. [ 4.828291] systemd[1]: Created slice Slice /system/serial-getty. [ 4.876696] systemd[1]: Created slice User and Session Slice. [ 4.925766] systemd[1]: Started Dispatch Password Requests to Console Directory Watch. [ 4.975677] systemd[1]: Started Forward Password Requests to Wall Directory Watch. [ 5.026282] systemd[1]: Reached target Host and Network Name Lookups. [ 5.075150] systemd[1]: Reached target Path Units. [ 5.115096] systemd[1]: Reached target Remote File Systems. [ 5.154873] systemd[1]: Reached target Slice Units. [ 5.195392] systemd[1]: Reached target Swaps. [ 5.361887] systemd[1]: Listening on RPCbind Server Activation Socket. [ 5.415355] systemd[1]: Reached target RPC Port Mapper. [ 5.466509] systemd[1]: Listening on Syslog Socket. [ 5.515740] systemd[1]: Listening on initctl Compatibility Named Pipe. [ 5.576147] systemd[1]: Journal Audit Socket was skipped because of a failed condition check (ConditionSecurity=audit). [ 5.591762] systemd[1]: Listening on Journal Socket (/dev/log). [ 5.636220] systemd[1]: Listening on Journal Socket. [ 5.675720] systemd[1]: Listening on Network Service Netlink Socket. [ 5.716474] systemd[1]: Listening on udev Control Socket. [ 5.756071] systemd[1]: Listening on udev Kernel Socket. [ 5.806179] systemd[1]: Listening on User Database Manager Socket. [ 5.856491] systemd[1]: Huge Pages File System was skipped because of a failed condition check (ConditionPathExists=/sys/kernel/mm/hugepages). [ 5.878363] systemd[1]: POSIX Message Queue File System was skipped because of a failed condition check (ConditionPathExists=/proc/sys/fs/mqueue). [ 5.955692] systemd[1]: Mounting Kernel Debug File System... [ 5.995288] systemd[1]: Kernel Trace File System was skipped because of a failed condition check (ConditionPathExists=/sys/kernel/tracing). [ 6.019735] systemd[1]: Mounting Temporary Directory /tmp... [ 6.065626] systemd[1]: Starting Create List of Static Device Nodes... [ 6.114085] systemd[1]: Starting Load Kernel Module configfs... [ 6.159790] systemd[1]: Starting Load Kernel Module drm... [ 6.204351] systemd[1]: Starting Load Kernel Module fuse... [ 6.236455] fuse: init (API version 7.37) [ 6.250904] systemd[1]: Starting RPC Bind... [ 6.289627] systemd[1]: Starting File System Check on Root Device... [ 6.343960] systemd[1]: Starting Journal Service... [ 6.356492] systemd[1]: Load Kernel Modules was skipped because all trigger condition checks failed. [ 6.375685] systemd[1]: Starting Generate network units from Kernel command line... [ 6.393680] systemd[1]: Starting Apply Kernel Variables... [ 6.410982] systemd[1]: Starting Coldplug All udev Devices... [ 6.436444] systemd[1]: Started RPC Bind. [ 6.445279] systemd[1]: Mounted Kernel Debug File System. [ 6.457154] systemd[1]: Mounted Temporary Directory /tmp. [ 6.469694] systemd[1]: Finished Create List of Static Device Nodes. [ 6.483106] systemd[1]: modprobe@configfs.service: Deactivated successfully. [ 6.495326] systemd[1]: Finished Load Kernel Module configfs. [ 6.506599] systemd[1]: modprobe@drm.service: Deactivated successfully. [ 6.518598] systemd[1]: Finished Load Kernel Module drm. [ 6.556448] systemd[1]: modprobe@fuse.service: Deactivated successfully. [ 6.567294] systemd[1]: Finished Load Kernel Module fuse. [ 6.606282] systemd[1]: Finished File System Check on Root Device. [ 6.646452] systemd[1]: Started Journal Service. [ 6.939126] EXT4-fs (mmcblk2p2): re-mounted. Quota mode: none. [ 7.138644] systemd-journald[211]: Received client request to flush runtime journal. [ 8.152598] fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW [ 8.214852] fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW [ 8.303576] coda 2040000.vpu: Firmware code revision: 46076 [ 8.312489] coda 2040000.vpu: Initialized CODA960. [ 8.314492] fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW [ 8.317398] coda 2040000.vpu: Firmware version: 3.1.1 [ 8.349669] coda 2040000.vpu: coda-jpeg-encoder registered as video9 [ 8.372591] coda 2040000.vpu: coda-jpeg-decoder registered as video10 [ 8.386575] coda 2040000.vpu: coda-video-encoder registered as video11 [ 8.407786] coda 2040000.vpu: coda-video-decoder registered as video12 [ 8.703619] imx-sdma 20ec000.dma-controller: loaded firmware 3.5 [ 8.772718] fsl-ssi-dai 2028000.ssi: No cache defaults, reading back from HW [ 9.034461] fsl-asoc-card sound: ASoC: driver name too long 'imx6q-apalis-sgtl5000' -> 'imx6q-apalis-sg' [ 9.384798] systemd-journald[211]: Oldest entry in /run/log/journal/b94b84a47d154d4ca716dddd2751c108/system.journal is older than the configured file retention duration (1month), suggesting rotation. [ 9.418841] systemd-journald[211]: /run/log/journal/b94b84a47d154d4ca716dddd2751c108/system.journal: Journal header limits reached or header out-of-date, rotating. [ 11.595267] using random self ethernet address [ 11.603353] using random host ethernet address [ 11.892865] cfg80211: Loading compiled-in X.509 certificates for regulatory database [ 11.927240] cfg80211: Loaded X.509 cert 'sforshee: 00b28ddf47aef9cea7' [ 11.941341] cfg80211: Loaded X.509 cert 'wens: 61c038651aabdcf94bd0ac7ff06c7248db18c600' [ 11.974313] usb0: HOST MAC 0a:8a:a4:fb:35:15 [ 11.983489] usb0: MAC 4e:5f:d3:fe:70:8b [ 12.268412] Micrel KSZ9031 Gigabit PHY 2188000.ethernet-1:07: attached PHY driver (mii_bus:phy_addr=2188000.ethernet-1:07, irq=58) [ 13.913975] fec 2188000.ethernet eth0: Link is Up - 100Mbps/Full - flow control rx/tx [ 13.925037] IPv6: ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready [ 13.952970] 8021q: 802.1Q VLAN Support v1.8 [ 42.384689] usb 1-1.3: new high-speed USB device number 3 using ci_hdrc [ 44.777462] usb 1-1.3: New USB device found, idVendor=046d, idProduct=082d, bcdDevice= 0.11 [ 44.786003] usb 1-1.3: New USB device strings: Mfr=0, Product=2, SerialNumber=1 [ 44.793476] usb 1-1.3: Product: HD Pro Webcam C920 [ 44.798464] usb 1-1.3: SerialNumber: 2E5C8B3F [ 44.921260] usb 1-1.3: Found UVC 1.00 device HD Pro Webcam C920 (046d:082d) [ 44.933107] input: HD Pro Webcam C920 as /devices/platform/soc/2100000.bus/2184200.usb/ci_hdrc.1/usb1/1-1/1-1.3/1-1.3:1.0/input/input1 [ 44.950209] usbcore: registered new interface driver uvcvideo [ 45.176050] usbcore: registered new interface driver snd-usb-audio [ 134.270517] usb 1-1.3: USB disconnect, device number 3 [ 136.250342] usb 1-1.3: new high-speed USB device number 4 using ci_hdrc [ 138.702958] usb 1-1.3: New USB device found, idVendor=046d, idProduct=082d, bcdDevice= 0.11 [ 138.711448] usb 1-1.3: New USB device strings: Mfr=0, Product=2, SerialNumber=1 [ 138.718872] usb 1-1.3: Product: HD Pro Webcam C920 [ 138.723795] usb 1-1.3: SerialNumber: 2E5C8B3F [ 138.742585] usb 1-1.3: Found UVC 1.00 device HD Pro Webcam C920 (046d:082d) [ 138.756428] input: HD Pro Webcam C920 as /devices/platform/soc/2100000.bus/2184200.usb/ci_hdrc.1/usb1/1-1/1-1.3/1-1.3:1.0/input/input2 [ 335.113223] usb 1-1.3: reset high-speed USB device number 4 using ci_hdrc
iMX6では、GPUにetnavivが利用されます。
root@apalis-imx6-10870254:~# dmesg | grep etnaviv [ 1.461276] etnaviv etnaviv: bound 130000.gpu (ops gpu_ops) [ 1.467157] etnaviv etnaviv: bound 134000.gpu (ops gpu_ops) [ 1.473020] etnaviv etnaviv: bound 2204000.gpu (ops gpu_ops) [ 1.478744] etnaviv-gpu 130000.gpu: model: GC2000, revision: 5108 [ 1.485062] etnaviv-gpu 134000.gpu: model: GC320, revision: 5007 [ 1.491152] etnaviv-gpu 2204000.gpu: model: GC355, revision: 1215 [ 1.497303] etnaviv-gpu 2204000.gpu: Ignoring GPU with VG and FE2.0 [ 1.504054] [drm] Initialized etnaviv 1.3.0 20151214 for etnaviv on minor 0
VPUにはcodaが利用されます。
root@apalis-imx6-10870254:~# dmesg | grep coda [ 8.303576] coda 2040000.vpu: Firmware code revision: 46076 [ 8.312489] coda 2040000.vpu: Initialized CODA960. [ 8.317398] coda 2040000.vpu: Firmware version: 3.1.1 [ 8.349669] coda 2040000.vpu: coda-jpeg-encoder registered as video9 [ 8.372591] coda 2040000.vpu: coda-jpeg-decoder registered as video10 [ 8.386575] coda 2040000.vpu: coda-video-encoder registered as video11 [ 8.407786] coda 2040000.vpu: coda-video-decoder registered as video12
最も大きな違いは、多くのビデオデバイスがあることです。V4L2サブシステムは、オープンソースとなったサブシステムをこのようにエクスポートします。
... root@apalis-imx6-10870254:~# ls -l /dev/video* crw-rw---- 1 root video 81, 0 Apr 28 2022 /dev/video0 crw-rw---- 1 root video 81, 1 Apr 28 2022 /dev/video1 crw-rw---- 1 root video 81, 24 Jan 30 09:27 /dev/video11 crw-rw---- 1 root video 81, 25 Jan 30 09:27 /dev/video12 crw-rw---- 1 root video 81, 26 Jan 30 09:27 /dev/video13 crw-rw---- 1 root video 81, 27 Jan 30 09:27 /dev/video14 crw-rw---- 1 root video 81, 2 Apr 28 2022 /dev/video2 crw-rw---- 1 root video 81, 3 Apr 28 2022 /dev/video3 crw-rw---- 1 root video 81, 4 Apr 28 2022 /dev/video4 crw-rw---- 1 root video 81, 5 Apr 28 2022 /dev/video5 crw-rw---- 1 root video 81, 6 Apr 28 2022 /dev/video6 crw-rw---- 1 root video 81, 7 Apr 28 2022 /dev/video7 crw-rw---- 1 root video 81, 21 Apr 28 2022 /dev/video8 (CONNECT USB CAMERA) ... [ 416.894702] usb 1-1.3: new high-speed USB device number 5 using ci_hdrc [ 419.337278] usb 1-1.3: New USB device found, idVendor=046d, idProduct=082d, bcdDevice= 0.11 [ 419.345830] usb 1-1.3: New USB device strings: Mfr=0, Product=2, SerialNumber=1 [ 419.353211] usb 1-1.3: Product: HD Pro Webcam C920 [ 419.358062] usb 1-1.3: SerialNumber: 2E5C8B3F [ 419.371425] usb 1-1.3: Found UVC 1.00 device HD Pro Webcam C920 (046d:082d) [ 419.386347] input: HD Pro Webcam C920 as /devices/platform/soc/2100000.bus/2184200.usb/ci_hdrc.1/usb1/1-1/1-1.3/1-1.3:1.0/input/input3 root@apalis-imx6-10870254:~# ls -l /dev/video* crw-rw---- 1 root video 81, 0 Apr 28 2022 /dev/video0 crw-rw---- 1 root video 81, 1 Apr 28 2022 /dev/video1 crw-rw---- 1 root video 81, 23 Jan 30 09:34 /dev/video10 crw-rw---- 1 root video 81, 24 Jan 30 09:27 /dev/video11 crw-rw---- 1 root video 81, 25 Jan 30 09:27 /dev/video12 crw-rw---- 1 root video 81, 26 Jan 30 09:27 /dev/video13 crw-rw---- 1 root video 81, 27 Jan 30 09:27 /dev/video14 crw-rw---- 1 root video 81, 2 Apr 28 2022 /dev/video2 crw-rw---- 1 root video 81, 3 Apr 28 2022 /dev/video3 crw-rw---- 1 root video 81, 4 Apr 28 2022 /dev/video4 crw-rw---- 1 root video 81, 5 Apr 28 2022 /dev/video5 crw-rw---- 1 root video 81, 6 Apr 28 2022 /dev/video6 crw-rw---- 1 root video 81, 7 Apr 28 2022 /dev/video7 crw-rw---- 1 root video 81, 21 Apr 28 2022 /dev/video8
USBカメラがvideo 9と10として認識されていることが分かります。
カメラの接続を外し、起動してもう一度試します。Video 13と14になりました。これは問題にはなりません。同じデバイスに割り当てる必要がある場合は、udevルールを使用します。
root@apalis-imx6-10870254:~# [ 44.777462] usb 1-1.3: New USB device found, idVendor=046d, idProduct=082d, bcdDevice= 0.11 [ 44.786003] usb 1-1.3: New USB device strings: Mfr=0, Product=2, SerialNumber=1 [ 44.793476] usb 1-1.3: Product: HD Pro Webcam C920 [ 44.798464] usb 1-1.3: SerialNumber: 2E5C8B3F [ 44.921260] usb 1-1.3: Found UVC 1.00 device HD Pro Webcam C920 (046d:082d) [ 44.933107] input: HD Pro Webcam C920 as /devices/platform/soc/2100000.bus/2184200.usb/ci_hdrc.1/usb1/1-1/1-1.3/1-1.3:1.0/input/input1 [ 44.950209] usbcore: registered new interface driver uvcvideo [ 45.176050] usbcore: registered new interface driver snd-usb-audio
カメラとハードウェアアクセラレーテッドHDMIにGstreamerを利用
Gstreamerは、Linuxにおけるビデオ処理に最高のツールです。
上記のコアについてGstreamerで直接確認しても、これについての言及はありません。これは、Gstreamerの外で、既にV4L2がすべてを処理しているからです。
root@apalis-imx6-10870254:~# gst-inspect-1.0 | grep vpu root@apalis-imx6-10870254:~# gst-inspect-1.0 | grep coda root@apalis-imx6-10870254:~# gst-inspect-1.0 | grep imx root@apalis-imx6-10870254:~# gst-inspect-1.0 | grep v4l2 video4linux2: v4l2convert: V4L2 Video Converter video4linux2: v4l2deviceprovider (GstDeviceProviderFactory) video4linux2: v4l2h264dec: V4L2 H264 Decoder video4linux2: v4l2h264enc: V4L2 H.264 Encoder video4linux2: v4l2jpegdec: V4L2 JPEG Decoder video4linux2: v4l2jpegenc: V4L2 JPEG Encoder video4linux2: v4l2mpeg2dec: V4L2 MPEG2 Decoder video4linux2: v4l2mpeg4dec: V4L2 MPEG4 Decoder video4linux2: v4l2mpeg4enc: V4L2 MPEG4 Encoder video4linux2: v4l2radio: Radio (video4linux2) Tuner video4linux2: v4l2sink: Video (video4linux2) Sink video4linux2: v4l2src: Video (video4linux2) Source
カメラの再生：
gst-launch-1.0 v4l2src device=/dev/video13 ! video/x-raw,width=1280,height=720,framerate=10/1 ! waylandsink sync=false
CPU使用率は非常に低度です。
VPUを利用してみましょう。
動画のデコーディング：
wget http://linode.boundarydevices.com/videos/trailer_1080p_h264_mp3.avi root@apalis-imx6-10870254:~# gst-launch-1.0 filesrc location=/home/root/trailer_1080p_h264_mp3.avi ! avidemux ! h264parse ! v4l2h264dec ! videoconvert ! waylandsink
これも問題なく動作します。
ハードウェアアクセラレーションは利用されているのでしょうか？
root@apalis-imx6-10870254:~# gst-inspect-1.0 v4l2h264dec
Factory Details:
Rank primary + 1 (257)
Long-name V4L2 H264 Decoder
Klass Codec/Decoder/Video/Hardware
Description Decodes H264 streams via V4L2 API
Author Nicolas Dufresne <nicolas.dufresne@collabora.com>
Plugin Details:
Name video4linux2
Description elements for Video 4 Linux
Filename /usr/lib/gstreamer-1.0/libgstvideo4linux2.so
Version 1.20.7
License LGPL
Source module gst-plugins-good
Source release date 2023-07-26
Binary package GStreamer Good Plug-ins source release
Origin URL Unknown package origin
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoDecoder
+----GstV4l2VideoDec
+----v4l2h264dec
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-h264
stream-format: byte-stream
alignment: au
level: { (string)1, (string)1b, (string)1.1, (string)1.2, (string)1.3, (string)2, (string)2.1, (string)2.2, (string)3, (string)3.1, (string)3.2, (string)4, (string)4.1 }
profile: { (string)constrained-baseline, (string)main, (string)high }
SRC template: 'src'
Availability: Always
Capabilities:
video/x-raw
format: { (string)NV12, (string)I420, (string)YV12, (string)YUY2 }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
automatic-request-sync-point-flags: Flags to use when automatically requesting sync points
flags: readable, writable
Flags "GstVideoDecoderRequestSyncPointFlags" Default: 0x00000003, "corrupt-output+discard-input"
(0x00000001): discard-input - GST_VIDEO_DECODER_REQUEST_SYNC_POINT_DISCARD_INPUT
(0x00000002): corrupt-output - GST_VIDEO_DECODER_REQUEST_SYNC_POINT_CORRUPT_OUTPUT
automatic-request-sync-points: Automatically request sync points when it would be useful
flags: readable, writable
Boolean. Default: false
capture-io-mode : Capture I/O mode (matches src pad)
flags: readable, writable
Enum "GstV4l2IOMode" Default: 0, "auto"
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap - GST_V4L2_IO_MMAP
(3): userptr - GST_V4L2_IO_USERPTR
(4): dmabuf - GST_V4L2_IO_DMABUF
(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT
device : Device location
flags: readable
String. Default: "/dev/video12"
device-fd : File descriptor of the device
flags: readable
Integer. Range: -1 - 2147483647 Default: -1
device-name : Name of the device
flags: readable
String. Default: null
discard-corrupted-frames: Discard frames marked as corrupted instead of outputting them
flags: readable, writable
Boolean. Default: false
extra-controls : Extra v4l2 controls (CIDs) for the device
flags: readable, writable
Boxed pointer of type "GstStructure"
max-errors : Max consecutive decoder errors before returning flow error
flags: readable, writable
Integer. Range: -1 - 2147483647 Default: 10
min-force-key-unit-interval: Minimum interval between force-keyunit requests in nanoseconds
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "v4l2h264dec0"
output-io-mode : Output side I/O mode (matches sink pad)
flags: readable, writable
Enum "GstV4l2IOMode" Default: 0, "auto"
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap - GST_V4L2_IO_MMAP
(3): userptr - GST_V4L2_IO_USERPTR
(4): dmabuf - GST_V4L2_IO_DMABUF
(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
qos : Handle Quality-of-Service events from downstream
flags: readable, writable
Boolean. Default: true
(END)
root@apalis-imx6-10870254:~# gst-inspect-1.0 v4l2h264enc
Factory Details:
Rank primary + 1 (257)
Long-name V4L2 H.264 Encoder
Klass Codec/Encoder/Video/Hardware
Description Encode H.264 video streams via V4L2 API
Author ayaka <ayaka@soulik.info>
Plugin Details:
Name video4linux2
Description elements for Video 4 Linux
Filename /usr/lib/gstreamer-1.0/libgstvideo4linux2.so
Version 1.20.7
License LGPL
Source module gst-plugins-good
Source release date 2023-07-26
Binary package GStreamer Good Plug-ins source release
Origin URL Unknown package origin
GObject
+----GInitiallyUnowned
+----GstObject
+----GstElement
+----GstVideoEncoder
+----GstV4l2VideoEnc
+----GstV4l2H264Enc
+----v4l2h264enc
Implemented Interfaces:
GstPreset
Pad Templates:
SINK template: 'sink'
Availability: Always
Capabilities:
video/x-raw
format: { (string)NV12, (string)I420, (string)YV12 }
width: [ 1, 32768 ]
height: [ 1, 32768 ]
framerate: [ 0/1, 2147483647/1 ]
SRC template: 'src'
Availability: Always
Capabilities:
video/x-h264
stream-format: byte-stream
alignment: au
level: { (string)1, (string)1b, (string)1.1, (string)1.2, (string)1.3, (string)2, (string)2.1, (string)2.2, (string)3, (string)3.1, (string)3.2, (string)4, (string)4.1, (string)4.2 }
profile: { (string)baseline, (string)constrained-baseline }
Element has no clocking capabilities.
Element has no URI handling capabilities.
Pads:
SINK: 'sink'
Pad Template: 'sink'
SRC: 'src'
Pad Template: 'src'
Element Properties:
capture-io-mode : Capture I/O mode (matches src pad)
flags: readable, writable
Enum "GstV4l2IOMode" Default: 0, "auto"
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap - GST_V4L2_IO_MMAP
(3): userptr - GST_V4L2_IO_USERPTR
(4): dmabuf - GST_V4L2_IO_DMABUF
(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT
device : Device location
flags: readable
String. Default: "/dev/video11"
device-fd : File descriptor of the device
flags: readable
Integer. Range: -1 - 2147483647 Default: -1
device-name : Name of the device
flags: readable
String. Default: null
extra-controls : Extra v4l2 controls (CIDs) for the device
flags: readable, writable
Boxed pointer of type "GstStructure"
min-force-key-unit-interval: Minimum interval between force-keyunit requests in nanoseconds
flags: readable, writable
Unsigned Integer64. Range: 0 - 18446744073709551615 Default: 0
name : The name of the object
flags: readable, writable, 0x2000
String. Default: "v4l2h264enc0"
output-io-mode : Output side I/O mode (matches sink pad)
flags: readable, writable
Enum "GstV4l2IOMode" Default: 0, "auto"
(0): auto - GST_V4L2_IO_AUTO
(1): rw - GST_V4L2_IO_RW
(2): mmap - GST_V4L2_IO_MMAP
(3): userptr - GST_V4L2_IO_USERPTR
(4): dmabuf - GST_V4L2_IO_DMABUF
(5): dmabuf-import - GST_V4L2_IO_DMABUF_IMPORT
parent : The parent of the object
flags: readable, writable, 0x2000
Object of type "GstObject"
qos : Handle Quality-of-Service events from downstream
flags: readable, writable
Boolean. Default: false
(END)
されています。鍵は、v4l2h264decとv4l2h264encが利用しているデバイスにあります。video11, video12への言及があり、これらは、VPUについてcodaがエクスポートするデバイスです。上記のスニペットと同じです。これは v4l2-ctl --list-devicesv4l2-ctlでも取得できます。
root@apalis-imx6-10870254:~# dmesg | grep coda [ 8.303576] coda 2040000.vpu: Firmware code revision: 46076 [ 8.312489] coda 2040000.vpu: Initialized CODA960. [ 8.317398] coda 2040000.vpu: Firmware version: 3.1.1 [ 8.349669] coda 2040000.vpu: coda-jpeg-encoder registered as video9 [ 8.372591] coda 2040000.vpu: coda-jpeg-decoder registered as video10 [ 8.386575] coda 2040000.vpu: coda-video-encoder registered as video11 [ 8.407786] coda 2040000.vpu: coda-video-decoder registered as video12
root@apalis-imx6-10870254:~# v4l2-ctl --list-devices imx-capture (platform:2400000.ipu): /dev/video1 /dev/video2 imx-capture (platform:2800000.ipu): /dev/video5 /dev/video6 imx-media (platform:capture-subsystem): /dev/media0 CODA960 (platform:coda): /dev/video9 /dev/video10 /dev/video11 /dev/video12 imx-csc-scaler (platform:imx-csc-scaler): /dev/video8 imx-capture (platform:imx-ipuv3-csi.0): /dev/video0 imx-capture (platform:imx-ipuv3-csi.1): /dev/video3 imx-capture (platform:imx-ipuv3-csi.4): /dev/video4 imx-capture (platform:imx-ipuv3-csi.5): /dev/video7 HD Pro Webcam C920 (usb-ci_hdrc.1-1.3): /dev/video13 /dev/video14 /dev/media1
MIPI CSIカメラ
Apalis iMX6では、デフォルトで追加される、ADV7280のようなCSIデバイスがいくつかあります。
MIPI CSIデバイスは、デバイスツリーでIPUが直接利用します。
root@apalis-imx6-10870254:~# dmesg | grep video* [ 0.211956] videodev: Linux video capture interface: v2.00 [ 2.464827] imx-ipuv3-csi imx-ipuv3-csi.0: Registered ipu1_csi0 capture as /dev/video0 [ 2.476479] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpenc capture as /dev/video1 [ 2.487808] imx-ipuv3 2400000.ipu: Registered ipu1_ic_prpvf capture as /dev/video2 [ 2.499175] imx-ipuv3-csi imx-ipuv3-csi.1: Registered ipu1_csi1 capture as /dev/video3 [ 2.510908] imx-ipuv3-csi imx-ipuv3-csi.4: Registered ipu2_csi0 capture as /dev/video4 [ 2.522422] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpenc capture as /dev/video5 [ 2.533685] imx-ipuv3 2800000.ipu: Registered ipu2_ic_prpvf capture as /dev/video6 [ 2.544953] imx-ipuv3-csi imx-ipuv3-csi.5: Registered ipu2_csi1 capture as /dev/video7 [ 3.427818] imx-media: Registered ipu_ic_pp csc/scaler as /dev/video8 [ 8.349669] coda 2040000.vpu: coda-jpeg-encoder registered as video9 [ 8.372591] coda 2040000.vpu: coda-jpeg-decoder registered as video10 [ 8.386575] coda 2040000.vpu: coda-video-encoder registered as video11 [ 8.407786] coda 2040000.vpu: coda-video-decoder registered as video12 [ 44.950209] usbcore: registered new interface driver uvcvideo
多くのimxデバイスがCSIキャプチャーに参照されているのがわかります。
Apalis iMX6デバイスツリー：imx6q-apalis-ixora-v1.2.dts → imx6q.dtsi (mx6qdl.dtsi) + imx6qdl-apalis.dtsi
ipu1: ipu@2400000
...
ipu1_csi0: port@0 {
reg = <0>;
ipu1_csi0_from_ipu1_csi0_mux: endpoint {
remote-endpoint = <&ipu1_csi0_mux_to_ipu1_csi0>;
};
};
ipu1_csi1: port@1 {
reg = <1>;
};
ipu2: ipu@2800000
...
ipu2_csi0: port@0 {
reg = <0>;
ipu2_csi0_from_mipi_vc2: endpoint {
remote-endpoint = <&mipi_vc2_to_ipu2_csi0>;
};
};
ipu2_csi1: port@1 {
reg = <1>;
ipu2_csi1_from_ipu2_csi1_mux: endpoint {
remote-endpoint = <&ipu2_csi1_mux_to_ipu2_csi1>;
};
};
...
capture-subsystem {
compatible = "fsl,imx-capture-subsystem";
ports = <&ipu1_csi0>, <&ipu1_csi1>, <&ipu2_csi0>, <&ipu2_csi1>;
};
...
&gpr {
ipu1_csi0_mux {
#address-cells = <1>;
#size-cells = <0>;
status = "disabled";
port@1 {
reg = <1>;
ipu1_csi0_mux_from_parallel_sensor: endpoint {
remote-endpoint = <&adv7280_to_ipu1_csi0_mux>;
};
};
};
};
...
&i2c3 {
clock-frequency = <100000>;
pinctrl-names = "default", "gpio";
pinctrl-0 = <&pinctrl_i2c3>;
pinctrl-1 = <&pinctrl_i2c3_gpio>;
scl-gpios = <&gpio3 17 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
sda-gpios = <&gpio3 18 (GPIO_ACTIVE_HIGH | GPIO_OPEN_DRAIN)>;
status = "disabled";
adv_7280: adv7280@21 {
compatible = "adi,adv7280";
adv,force-bt656-4;
pinctrl-names = "default";
pinctrl-0 = <&pinctrl_ipu1_csi0>;
reg = <0x21>;
status = "disabled";
port {
adv7280_to_ipu1_csi0_mux: endpoint {
bus-width = <8>;
remote-endpoint = <&ipu1_csi0_mux_from_parallel_sensor>;
};
};
};
ADIによると、drivers/media/i2cにあるドライバーを利用する必要があります。これはV4L2と互換性があります。
ただし、他にデバイスツリーオーバーレイを有効にする必要があります。
apalis-imx6_adv7280_overlay.dtboを追加するだけで機能するはずです。
Apalis iMX6では、/boot/overlays.txtに追加します。
https://git.toradex.com/cgit/device-tree-overlays.git/tree/overlays/apalis-imx6_adv7280_overlay.dts
これで、カーネルモジュールadv7180が追加されます。
ADV7180は、デフォルトでimx-ipuv3-csi imx-ipuv3-csi.0:Registered ipu1_csi0 capture as /dev/video0を使っているはずです。
imx-captureはdrivers/staging/media/imx/imx-media-capture.cからです。
CONFIG_VIDEO_IMX_CSIとCONFIG_VIDEO_IMX_MEDIAは、構成で有効化されています。
こちらで説明されているとおりコンフィグレーションを適用する際は（Artifactoryからconfigファイルを取得）、フォルダー全体がコンパイルされます。
imx-media-capture.cを確認すると、実質的にはメディア関連のものにV4L2を利用することが分かります。
これはつまり、ADV7180/7280のすべてのコンポーネントはV4L2と互換性があるということです。
一旦adv7280デバイスツリーオーバーレイが有効化され、I2Cアドレスが正しく設定されていれば（0x21？）Gstreamerは機能するはずです。dmesgを使い、ipu1_csi0でビデオデバイスを照合してください。
[ 2.464827] imx-ipuv3-csi imx-ipuv3-csi.0: Registered ipu1_csi0 capture as /dev/video0
Author:
Alvaro Garcia（Toradex Japan株式会社）
Alvaro Garcia（Toradex Japan株式会社）
Leave a comment
Please login to leave a comment!
Latest Blog
Friday, April 5, 2024
Tuesday, March 19, 2024
Friday, January 12, 2024
