How to prevent the ever changing downloads tarballs? AUTOINC?

Technical Support
, ,
1

Hello,

from time to time I encounter these very annoying fetch errors:

ERROR: gnu-config-native-20150728+gitAUTOINC+b576fa87c1-r0 do_fetch: Fetcher failure for URL: 'git://git.sv.gnu.org/config.git'. Unable to fetch URL from any source.
ERROR: gnu-config-native-20150728+gitAUTOINC+b576fa87c1-r0 do_fetch: Function failed: base_do_fetch
ERROR: Logfile of failure stored in: /home/user/workspace-kernel/yocto/build/tmp-glibc/work/x86_64-linux/gnu-config-native/20150728+gitAUTOINC+b576fa87c1-r0/temp/log.do_fetch.5123
ERROR: Task (virtual:native:/home/user/workspace-kernel/yocto/build/../sources/openembedded-core/meta/recipes-devtools/gnu-config/gnu-config_git.bb:do_fetch) failed with exit code '1'

Why is bitbake unsatisfied with the established downloads?

Regards
Dirk

2

Just to explain the environment:
I generate tarballs out of the fetched libs:
BB_GENERATE_MIRROR_TARBALLS = “1”
BB_NO_NETWORK = “0”
CONNECTIVITY_CHECK_URIS=“”
DISABLE_NETWORK_SANITY = “1”
these downloads tarballs are archived within a new git repo to have reproducible builds.

3

Is the log file log.do_fetch.5123 maybe telling more?

Is there a git2_git.sv.gnu.org.config.git.tar.gz in the archived tarball directory?

I would assume that it should work when metadata did not change…

4

Hi,
thank you for your answer.

  1. the do_fetch tells that a fetch fails, that a tarball has been saved, that another fetch failslink text
  2. the gnu tarball is fetched weeks ago and archived for reuse (unpacked before each build). But the do_fetch now deletes this tar.gz and tries to fetch a new one.

kind regards
Dirk

5

Ok, I understand now, in this step you really try to fetch the tarball from the network, not from a local tarball.

It really seems that fetching just fails. git.sv.gnu.org recently moved to git.savannah.gnu.org. There is already a patch for that in rocko, not sure whether that will get backported to morty.

Are you using INHERIT += "toradex-mirrors" in your local.conf? This should add Toradex source mirrors to the mirros list which should have a tarball of said git repository.

6

Thanks for your kind response,

in fact I’ve checked the availability of the network tarball and (desperately) changed the mirror within its gnu recipe.
But:

  1. There are more recipes than this one which have changed recently

  2. I would prefer that bitbake NEVER tries the network mirror once the local tarballs are established.

  3. But even the switch BB_NO_NETWORK = “1” still leads to network access trials: gnu-config-native-20150728+gitAUTOINC+b576fa87c1-r0 do_fetch: Network access disabled through BB_NO_NETWORK (or set indirectly due to use of BB_FETCH_PREMIRRORONLY) but access requested with command LANG=C git -c core.fsyncobjectfiles=0 fetch -f --prune --progress git://git.sv.gnu.org/config.git refs/:refs/ (for url git://git.sv.gnu.org/config.git)

  4. inherit += “toradex-mirrors” at least fixes the ever changing public git servers

kind regards

Dirk

7

In a comment to the original question you did write that you used BB_NO_NETWORK = "0"…

As far as I understand, BB_NO_NETWORK = "1" should really not use the network, at least that is what the documentation of BB_NO_NETWORK says.

8

BB_NO_NETWORK = “1” blocks effectively the network access, but gnu complains that it would like to fetch online resources although the tarball is already locally available. So I assume an error within the recipe which always tries t fetch the hottest and newest branch / tag / whatsoever

9

I doubt that this conclusion is correct. You write above you changed the mirror in the recipe. I guess with that the recipe and your local tarballs (which have been generated before the mirror update) do not match up anymore.

As far as I understand bitbake uses the mirror name/url to locate the tarball. So when you update the mirror in a recipe, you really need to regenerate tarballs…

10

In fact changing the mirror in the recipe has only been a test to check what is going on. I reverted it immediately because I only want to find out why the network is accessed at all.
So because everything is reverted: clean checkout of the recipes, artefacts (deploy, tmp-glibc. cache,…) deleted, fresh checkout of the local tarballs - the question remains: Why tries e.g. gnu and pk_config to update its tarballs?

11

Hi

For me it worked. I did the following:

Fresh checkout of 2.7 meta data:

...
repo init -u http://git.toradex.com/toradex-bsp-platform.git -b LinuxImageV2.8
repo sync
. export

Then added your configuration to additionally create the source tarballs to conf/local.conf and built only gnu-config-native:

$ head conf/local.conf
BB_GENERATE_MIRROR_TARBALLS = "1"
BB_NO_NETWORK = "0"
CONNECTIVITY_CHECK_URIS=""
DISABLE_NETWORK_SANITY = "1"

$ MACHINE=apalis-tk1 bitbake gnu-config-native

Then delete all the generated output and move to the no network setup with the source tarballs in place:

$ head conf/local.conf
BB_NO_NETWORK = "1"

#BB_GENERATE_MIRROR_TARBALLS = "1"
#BB_NO_NETWORK = "0"
#CONNECTIVITY_CHECK_URIS=""
#DISABLE_NETWORK_SANITY = "1"

$ rm -rf bitbake.lock  buildhistory  cache  sstate-cache  tmp-glibc downloads/git2

$ tree
.
├── conf
│   ├── bblayers.conf
│   └── local.conf
└── downloads
    ├── git2_git.sv.gnu.org.config.git.tar.gz
    ├── git2_git.sv.gnu.org.config.git.tar.gz.done
    ├── quilt-0.64.tar.gz
    └── quilt-0.64.tar.gz.done

2 directories, 6 files

With that fetching and building gnu-config-native successeeds with only the source tarballs:

$ MACHINE=apalis-tk1 bitbake gnu-config-native

$ grep "tar -x"  tmp-glibc/work/x86_64-linux/gnu-config-native/20150728+gitAUTOINC+b576fa87c1-r0/temp/log.do_fetch
DEBUG: Running export DBUS_SESSION_BUS_ADDRESS="unix:path=/run/user/1003/bus"; export PATH="..."; export HOME="..."; tar -xzf .../build/downloads/git2_git.sv.gnu.org.config.git.tar.gz

What do you do differently?

Max