Built the standard TorizonCore images (for 5.0.0), which took hours and hours
Added my local layer using bitbake-layers create-layer
Added my layer to conf/bblayers.conf
Created a “hello world” recipe in my layer
Run bitbake-layers show-recipes and confirmed my recipes are present
Run bitbake my-target and build my target successfully with no error messages
However, when I then attempt to build the TorizonCore images with my component added by running bitbake torizon-core-docker, I get one or more error messages as below (exact number of times I see this does vary somewhat from run to run):
NOTE: Executing Tasks
WARNING: Failed to get layers information. Exception: <class 'bb.process.ExecutionError'>
WARNING: torizon-core-docker-1.0-r0 do_rootfs: Failed to get layers information. Exception: <class 'bb.process.ExecutionError'>
WARNING: Failed to get layers information. Exception: <class 'bb.process.ExecutionError'>
WARNING: Failed to get layers information. Exception: <class 'bb.process.ExecutionError'>
and the build process torizon-core-docker.bb:do_image_ostreecommit task fails with numerous subsequent errors.
If I remove my layer from bblayers.conf, then the build succeeds every time, so it is clearly something wrong with my layer.
How can I debug this? How can I see what the underlying bb.process.ExecutionError exception is showing?
After several hours of adding asserts to the bitbake python libraries, it seems the failing command is running git rev-parse --abbrev-ref HEAD on the module directory. This fails in the demo case, as the layer is not a GIT module, it’s just a directory. This is called from layers/openembedded-core/meta/classes/metadata_scm.bbclass
Workaround is easy enough in this demo case, set the module up as a fake GIT project
However, for a real build our local code will likely be in SVN not GIT. And in a Docker-ised build, a layer might not look like it’s in SCM at all, it might be copied from the Docker context.
Is subversion (or any SCM other than GIT, or no SCM at all) supported for bitbake layers? Looking at the code, calls to git are spread all through the libraries so I suspect the answer is NO.
I’ve read as much documentation as I can find, but I have seen nothing that addresses this issue. Plenty about using SVN in recipes, but not about using SVN (or anything other than git, including plain directories) in layers.
As I read it, create-layer builds the layer framework including the conf/layer.conf file. add-layer just inserts the path in bblayers.conf, which I have done by hand (as recommended by Torazon here)
Even if my layer is totally empty of recipes, the build fails unless the layer looks like a GIT repo. For now, I can just create a sham git repo in the layer directory, as outlined above. This confuses the heck out of the IDE but otherwise seems to work.
Just so I understand on my side, so the issue is that your meta-layer itself is an SVN? This is interesting, while I don’t use SVN myself I wouldn’t think this a problem, though I may be wrong. Also I don’t quite understand your comment on not being able to use plain directories. For quick tests I’ve created meta-layers out of just plain directories before and it has never been a problem. Apologies if I misunderstand the situation.
So it seems everything works until you add your “component” to your layer correct? Could you maybe describe this component if possible? Like how you are adding it to your layer and such?
As far as I can see, the problem is that the bitbake framework requires that the layer directory itself look like a GIT repo, on which git rev-parse HEAD can run successfully. If the layer directory is an SVN project, or not in any SCM at all (which is what I mean by “plain directory”), then the build fails.
Here’s how I can replicate the problem (which basically follows the Torizon document):
Set up a dev environment until bitbake torizon-core-docker succeeds
bitbake-layers create-layer layers/mylayer
bitbake-layers add-layer layers/mylayer
bitbake torizon-core-docker now rebuilds the kernel (due to new recipe), and fails. Initially, with many warnings like
WARNING: /workdir/build-torizon/conf/…/…/layers/openembedded-core/meta/recipes-core/images/core-image-minimal-dev.bb: Failed to get layers information. Exception: <class ‘bb.process.ExecutionError’>
as the recipes are re-parsed, then the actual build fails with
WARNING: Failed to get layers information. Exception: <class 'bb.process.ExecutionError'>
WARNING: torizon-core-docker-1.0-r0 do_image_ostreecommit: Failed to get layers information. Exception: <class 'bb.process.ExecutionError'>
ERROR: torizon-core-docker-1.0-r0 do_image_ostreecommit: Execution of '/workdir/build-torizon/tmp-torizon/work/apalis_imx8-tdx-linux/torizon-core-docker/1.0-r0/temp/run.do_image_ostreecommit.290621' failed with exit code 1:
error: Parsing oe.layers=None : 0:expected value
WARNING: exit code 1 from a shell command.
If I turn the layers/mylayer directory into a git repo:
After some internal discussion, I believe your best course of action would just be work-around this limitation as you already are. For the time being we choose to support solely git as the versioning system with OSTree.
Though like I said you can continue to use SVN, but will need to work around this. Other than what you’ve already done, you can also try mirrioring your SVN repo to git for future development. Then like I said previously if you don’t plan to use or care about the OTA features of Torizon then it doesn’t matter too much how you work around this.
Thanks Jeremias. We do not need OTA updates, but for now we will proceed as above by using a fake GIT repo. Hopefully we can migrate to GIT at some point.
You might want to ask your documentation colleagues to update the tech note with this information.
Oh now that I see the full error I know what’s going on now. So actually this has nothing to do with the bitbake framework. Rather the issue is sort of our fault.
To explain, we include this package “OSTree” into our Torizon image. This package serves as the framework for our OTA update features. Whenever a Yocto build is done we create an OSTree update package out of that build for compatibility. However in this update package includes information about the build, namely information about the each layer revision, which we parse with git as you can tell.
Let me bring this up to the team and see if there are any thoughts on making our OTA update framework compatible with other version system than git. Worst-case scenario you could also just remove the OSTree components from the Yocto build. However I can’t really say how this might end up as this package is fairly interwoven into our build and other tooling. Really it might be more trouble than it’s worth, but still a possibility. In any case let me get back to you after speaking with the team internally.
I can add another, perhaps more manageable solution to this problem.
Install the git-svn package and then use git svn clone <SVNURL> <layerdir> command. This makes the layer directory look like a git repo, but the actual data is still in the SVN repo.
