Apalis TK1 mainline kernel build PCIe link fails

I have built a working system with the Apalis TK1 LT4 3.10.40 sources with a Debian jessie root file system.
The PCIe bus is functional with TK1 LT4 image but not with the mainline build of 4.9 kernel. With the mainline Linux kernel the system boots but the PCIe bus is not connecting, no link.

Are there any changes required to the kernel sources or DTS files for the TK1?

Apalis TK1 LT4 3.10.40

[    0.600210] PCIE.C: tegra_pcie_enable_regulators : regulator hvdd_pex
[    0.600311] PCIE.C: tegra_pcie_enable_regulators : regulator pexio
[    0.600403] PCIE.C: tegra_pcie_enable_regulators : regulator avdd_plle
[    1.306007] PCIE: port 0: link down, retrying
[    1.708212] PCIE: port 0: link down, retrying
[    2.110412] PCIE: port 0: link down, retrying
[    2.312488] PCIE: port 0: link down, ignoring
[    2.326670] PCI host bridge to bus 0000:00
[    2.326699] pci_bus 0000:00: root bus resource [mem 0x32100000-0x3fffffff]
[    2.326729] pci_bus 0000:00: root bus resource [mem 0x12100000-0x320fffff pref]
[    2.326760] pci_bus 0000:00: root bus resource [io  0x1000-0xffff]
[    2.326808] pci_bus 0000:00: No busn resource found for root bus, will use [bus 00-ff]
[    2.327209] PCI: bus0: Fast back to back transfers disabled
[    2.327238] pci 0000:00:00.0: bridge configuration invalid ([bus 00-00]), reconfiguring
[    2.329853] PCI: bus1: Fast back to back transfers disabled
[    2.330197] pcieport 0000:00:00.0: Signaling PME through PCIe PME interrupt
[    2.330227] pci 0000:01:00.0: Signaling PME through PCIe PME interrupt
[    2.330474] pcieport 0000:00:00.0: BAR 8: assigned [mem 0x32100000-0x321fffff]
[    2.330508] pcieport 0000:00:00.0: BAR 7: assigned [io  0x1000-0x1fff]
[    2.330540] pci 0000:01:00.0: BAR 0: assigned [mem 0x32100000-0x3211ffff]
[    2.330575] pci 0000:01:00.0: BAR 3: assigned [mem 0x32120000-0x32123fff]
[    2.330608] pci 0000:01:00.0: BAR 2: assigned [io  0x1000-0x101f]
[    2.330639] pcieport 0000:00:00.0: PCI bridge to [bus 01]
[    2.330666] pcieport 0000:00:00.0:   bridge window [io  0x1000-0x1fff]
[    2.330695] pcieport 0000:00:00.0:   bridge window [mem 0x32100000-0x321fffff]
[    2.330737] PCIE: No Link speed change happened

Linux mainline 4.9.79

[    3.188768] tegra-pcie 1003000.pcie-controller: link 0 down, retrying
[    3.617905] tegra-pcie 1003000.pcie-controller: link 0 down, retrying
[    4.046984] tegra-pcie 1003000.pcie-controller: link 0 down, retrying
[    4.055596] tegra-pcie 1003000.pcie-controller: link 0 down, ignoring
[    4.062075] tegra-pcie 1003000.pcie-controller: probing port 1, using 1 lanes
[    4.071413] tegra-pcie 1003000.pcie-controller: Slot present pin change, signature: 00000000
[    4.491905] tegra-pcie 1003000.pcie-controller: link 1 down, retrying
[    4.921031] tegra-pcie 1003000.pcie-controller: link 1 down, retrying
[    5.350114] tegra-pcie 1003000.pcie-controller: link 1 down, retrying
[    5.358721] tegra-pcie 1003000.pcie-controller: link 1 down, ignoring
[    5.365760] tegra-pcie 1003000.pcie-controller: PCI host bridge to bus 0000:00
[    5.373020] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
[    5.379210] pci_bus 0000:00: root bus resource [mem 0x13000000-0x1fffffff]
[    5.386108] pci_bus 0000:00: root bus resource [mem 0x20000000-0x3fffffff pref]
[    5.393449] pci_bus 0000:00: root bus resource [bus 00-ff]
[    5.398966] PCI: bus0: Fast back to back transfers enabled

Are you talking about our mainline BSP or you did your own vanilla mainline build? As we do have a bunch of specific patches on top. Also could you please post complete log files rather than just nonsensical snippets?

Hi Marcel,

My bad as they say for a general question at the start. I mistakenly thought you would assume I have reviewed the Toradex website and forums for all information as stated by the forum notes.

To clarify, I am using the information as supplied from the Toradex website.

and

https://developer.toradex.com/knowledge-base/apalis-tk1-mainline-with-nouveau

  1. Is the Toradex website information correct?

  2. Are the mainline git repositories the ones to fetch from per the Toradex web site?

  3. Do the DTS patches apply correctly to the TK1?

  4. We could use the LT4 3.10.40 kernel but would very much prefer to work on current Linux kernel since the 3.10 kernel is quite out of date.

I have attached the complete kernel boot logs as requested.

Regards,

  1. Is the Toradex website information correct?

Yes, at least it was back at the time of publication however both hardware as well as software may have evolved since back then.

  1. Are the mainline git repositories the ones to fetch from per the Toradex web site?

Yes, however I initially missed mentioning those additional patches required. The article got updated pending internal review.

  1. Do the DTS patches apply correctly to the TK1?

I’m not exactly sure what you are trying to get at with that question. Some patches are DTS related and hopefully do apply, yes. But other patches are Ethernet or PCIe driver source code related. Of course any patch is taken against a certain version and whether or not it still applies at a later time depends on what exactly changed. Usually patches won’t break across minor updates unless of course such update addressed very specific stuff also touched by the patch.

  1. We could use the LT4 3.10.40 kernel but would very much prefer to work on current Linux kernel since the 3.10 kernel is quite out of date.

Understood however please keep in mind that various NVIDIA proprietary features like CUDA or hardware accelerated multimedia are only available in their supported downstream version.

Hi Marcel,

Thank you for the reply, this helped make is clear where to look. The primary issue is due to the PCIe and LAN reset control functions in the PCIe driver for the Apalis TK1. These are not in the upstream kernel.

By hacking the mainline PCIe kernel driver with the reset control functions from the Toradex driver I now have the TK1 PCIe interface up on the 4.9.79 mainline kernel.

There is still some more work to do, but Debian 8 root file system and 4.9.79 kernel with the nouveau graphics drivers are now running.

Regards,

Thank you for the reply, this helped make is clear where to look.

You are very welcome.

The primary issue is due to the PCIe and LAN reset control functions in the PCIe driver for the Apalis TK1. These are not in the upstream kernel.

Yes, this is caused by a rather unfortunate combination of hardware limitations resp. erratas which makes upstreaming rather tricky.

By hacking the mainline PCIe kernel driver with the reset control functions from the Toradex driver I now have the TK1 PCIe interface up on the 4.9.79 mainline kernel.

Very good.

There is still some more work to do, but Debian 8 root file system and 4.9.79 kernel with the nouveau graphics drivers are now running.

Great.