VF61 dual ethernet eth1 inopertive after restart

Running unmodified Linux console image 2.8.6 on VF61 with device tree vf610-colibri-dual-eth.dts, both ethernet ports eth0 and eth1 appear when booting Linux at power up. LEDs on both ports light when plugging in cable, and both ports show valid IPV4 addresses when I list status with ifconfig (static IP address have been set using connmanctl).

However, when I restart the system form the command prompt using “shutdown -r now”, after rebooting I find that eth1 is completely inoperable. The LEDs for eth1 will not go on when I plug in a cable. Ifconfig shows no IPV4 address for eth1. Connmanctl services shows only the eth0 interface. I can do ifconfig eth1 down, and the eth1 interface disappears from the ifconfig list. I can do ifconfig eth1 up, and eth1 re-appears in the ifconfig list, but still shows no IP addresses. I cannot seem to make eth1 function at all after restarting using shutdown -r now. The only way I can get eth1 to function again is by powering off and powering on again.

Why does eth1 (the built-in ethernet port) not function after a restart? The eth0 port seems to come back up fine (at least as far as the LEDs indicating a cable plugged in, and ifconfig showing its IPV4 address). I would expect both ports to be functional after a restart just like they are after cycling power off and back on. Shouldn’t they boot up to the same state either way? I need both ports to function after a restart, which I can do remotely. I cannot cycle the power off and back on remotely.

How did you set up your board to use vf610-colibri-dual-eth.dts instead of regular one? May be it just replaced by original Device tree after restart?

Alex,

I configured my module to use vf610-colibri-dual-eth.dts as follows:

  1. I compiled the vf610-colibri-dual-eth.dts file to create a vf610-colibri-dual-eth.dtb binary file.
  2. I copied the dtb file to the colibri_vf folder on my SD card containing the Toradex Linux operating system.
  3. I renamed the vf610-colibri-dual-eth.dtb file to vf610-colibri-eval-v3.dtb (the default .dtb file name).
  4. I powered up the evaluation board with my SD card and VF61 module, and hit space to stop the boot process and enter the U Boot command prompt.
  5. I executed the following commands: “run setupdate”, “run prepare_ubi”, “run update_fdt”, “reset”.

No, the device tree file is NOT getting replaced by the original device tree file when I do “shutdown -r now”. That is NOT the way device tree files work. I find eth1 to be inoperable after an operating system restart. But if I then power cycle the board and let it boot up again, I find that both ethernet ports work fine again. The eth1 port is always inoperable after a “shutdown -r now” operating system restart, but always functions perfectly after a power off/of cycle to cause a hard (power on) restart of the operating system.

Give me a little credit here, I know what I am doing as far as replacing device tree files. Please try this yourself. Trust me, dual ethernet does not work after a software restart.

I am still waiting for an answer on this problem. I need to have both ethernet ports working after a software restart (shutdown -r now) because I cannot cycle power on my system remotely. You should be able to duplicate this problem yourself, since it happens with an unmodified Linux console image and an unmodified Toradex dual ethernet device tree file.

hi @irbsd

Sorry for the delayed answer. I will try to reproduce this issue on my side and come back to you within this week.

Best regards,
Jaski

Hi @irbsd

I tested Bsp 2.8b6 with the dual-eth devicetree on Colibri Evaluation Board and rebooted several times using shutdown -r now and the eth1 port was always working.

Did you test this on your custom carrier board?
If yes, could you share the serial bootlog for working and non working eth1 port.

Thanks and best regards,
Jaski

Running unmodified Tordex console Linus 2.8.6 with unmodified Toradex dual ethernet device tree file vf610-colibri-dual-eth.dts I captured the complete console output while doing the following:

  1. Power on system for a cold boot
  2. log in
  3. ifconfig
  4. connmanctl services
  5. shutdown -r now
  6. log in
  7. ifconfig
  8. connmanctl services

After the first cold boot eth1 was working properly.

After the soft restart eth1 was not functioning.

I have determined that eth1 is not always inoperable after a soft restart, but it happens often enough to be a serious problem.

The complete console output is attached.link text

Hi @irbsd

Thanks for the file. I will come within this week back to you.

Best regards,
Jaski

Hi @irbsd

For me, this sounds like you are facing an issue that is caused by a problem of the KSZ8041 when going into power-saving mode. There is an errata available which describes the issue. It can be found here.

Basically, the problem is that after recovery from a power-saving mode, the Ethernet PHY is not working properly. One workaround is not setting the PHY into the power-saving mode or doing a power cycle after recovering. On some of the Colibri modules, we are able to individually control the power rail of the Ethernet PHY. This resolves the problem. However, Colibri Vybrid does not have this feature.

To quote the errata you reference:

These errata are actually NXP errata affecting all i.MX and Vybrid processors. There are two issues in the boot rom when using the processors in a security enabled configuration (SEC_CONFIG[1] eFUSE is programmed). By default, this fuse is not programmed on Toradex modules. Customers not fusing this setting are therefore not affected by these issues.

  1. Since I have not modified the SEC_CONFIG eFUSE and do not use the VF61 module in a security enabled configuration, errata #1 does not apply to me.

  2. Since I am performing a software restart of the Linux operating system by executing “shutdown -r now”, and am never enabling any power save mode on my system, errata #2 does not apply to me.

Nevertheless, ethernet port eth1 becomes unusable after a software restart of the Linux operating system.

HI @irbsd

Thanks for your input.

Actually we cannot find the root-cause of your issue. Therefore we checked again your schematic and layout. One thing which is done bad is the separation EGND from GND using FB2. Could you try to replace FB2 with 0 Ohm and check if this solves the issue.

If not, then I am afraid that we cannot help you more on this issue without having the hardware.

Best regards,
Jaski

Why should you think it is bad to separate EGND from GND using FB2? I followed exactly what was done on the Toradex Colibri Evaluation Board. I have attached a copy of the evaluation board schematic here: link text

Please refer to sheet 8 of 23 of the attached schematic. You will see an L11 (rated at 220 ohms at 100 MHz) separating GND from ETH_AGND. I implemented EXACTLY the same filtering circuit on my board as Toradex did on the evaluation board. Why would you say this is a bad thing to do? Did Toradex do a bad thing on the evaluation board?

Hi @irbsd

Unfortunately, the your schematics is not 100% the same. In your schematics, the module pin 191 goes to the regular ground while on the Evaluation Board, it is connected to the ETH_AGND. The ETH_AGND is meant to be the reference plane underneath the high-speed signals as the return path.
Looking at your design, you use the regular GND as reference plane for these signals. The problem is that on the connector side (The circuit with the two resistors and the capacitor) are connected to a different net. Therefore, you do not have a good return path.

The current circuit on the Evaluation Board is not perfect. Most of the modules connect the SODIMM pin 191 directly to the regular ground. Therefore, it is actually better to get rid of the ETH_AGND and connect it directly to the ground. We added a note in the schematics. However, it is a bit misleading. It is meant to connect everything to ground, not only the pin 191 itself.

On the other hand, I do not think the ground is causing the issue on your board. Your traces are rather short on the carrier board. This means the return path is less a concern in this case. Also, the fact that the issue only appears after a software restart, I doubt that it is actually a layout issue. Nevertheless, I recommend doing a short test by removing the bead and shorting the pins.