ftrace on imx7

Technical Support
, , ,
1

I would like to enable FTRACE on the colibri-imx7 on the console-trdx-image. Initially I tried to add the “tools-profile” to the EXTRA_IMAGE_FEATURES as described here:

https://wiki.yoctoproject.org/wiki/Tracing_and_Profiling

However the recipe didn’t provide anything to build tools-profile for the console image. I then tried modifying the kernel configuration manually to have the proper flags, something approximatly like this:

CONFIG_NOP_TRACER=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_HAVE_C_RECORDMCOUNT=y
CONFIG_TRACE_CLOCK=y
CONFIG_RING_BUFFER=y
CONFIG_EVENT_TRACING=y
CONFIG_CONTEXT_SWITCH_TRACER=y
CONFIG_TRACING=y
CONFIG_TRACING_SUPPORT=y
CONFIG_FTRACE=y
# CONFIG_FUNCTION_TRACER is not set
# CONFIG_IRQSOFF_TRACER is not set
# CONFIG_SCHED_TRACER is not set
CONFIG_ENABLE_DEFAULT_TRACERS=y
# CONFIG_FTRACE_SYSCALLS is not set
# CONFIG_TRACER_SNAPSHOT is not set
CONFIG_BRANCH_PROFILE_NONE=y
# CONFIG_PROFILE_ANNOTATED_BRANCHES is not set
# CONFIG_PROFILE_ALL_BRANCHES is not set
# CONFIG_STACK_TRACER is not set
# CONFIG_BLK_DEV_IO_TRACE is not set
# CONFIG_UPROBE_EVENT is not set
# CONFIG_PROBE_EVENTS is not set
# CONFIG_TRACEPOINT_BENCHMARK is not set
# CONFIG_RING_BUFFER_BENCHMARK is not set
# CONFIG_RING_BUFFER_STARTUP_TEST is not set
# CONFIG_TRACE_ENUM_MAP_FILE is not set

Unfortunately the kernel that I built doesn’t populate the tracing directory in /sys/kernel/debug.

I issued the following commands to build the manually configured kernel:

bitbake console-trdx-image
bitbake -c menuconfig linux-toradex # added the configuration flags
bitbake -c compile -f linux-toradex
bitbake -c deploy linux-toradex
bitbake console-trdx-image

What am I missing on enabling FTRACE?

2

Please try with the following sequence of operations

MACHINE=colibri-imx7 bitbake -c menuconfig virtual/kernel
MACHINE=colibri-imx7 bitbake -f  -c compile virtual/kernel
MACHINE=colibri-imx7 bitbake console-trdx-image

I tried the above for an angstrom-lxde-image and had the tracing enabled correctly.

root@colibri-imx7:~# uname -a
Linux colibri-imx7 4.1.35-v2.7b1+gc117783 #1 SMP Wed Feb 8 06:23:46 CET 2017 armv7l armv7l armv7l GNU/Linux
root@colibri-imx7:~# cat /etc/issue
.---O---.                                           
|       |                  .-.           o o        
|   |   |-----.-----.-----.| |   .----..-----.-----.
|       |     | __  |  ---'| '--.|  .-'|     |     |
|   |   |  |  |     |---  ||  --'|  |  |  '  | | | |
'---'---'--'--'--.  |-----''----''--'  '-----'-'-'-'
                -'  |
                '---'

The Angstrom Distribution \n \l

Angstrom v2016.12 - Kernel \r

Colibri_iMX7_LinuxImageV2.7_20170208

root@colibri-imx7:~# ls /sys/kernel/debug/tracing/
README                      events                    saved_cmdlines_size  stack_trace         tracing_max_latency
available_events            free_buffer               set_event            stack_trace_filter  tracing_on
available_filter_functions  function_profile_enabled  set_ftrace_filter    trace               tracing_thresh
available_tracers           instances                 set_ftrace_notrace   trace_clock         uprobe_events
buffer_size_kb              max_graph_depth           set_ftrace_pid       trace_marker        uprobe_profile
buffer_total_size_kb        options                   set_graph_function   trace_options
current_tracer              per_cpu                   set_graph_notrace    trace_pipe
dyn_ftrace_total_info       printk_formats            snapshot             trace_stat
enabled_functions           saved_cmdlines            stack_max_size       tracing_cpumask

Also make sure that your kernel config changes are actually picked up. Check the output of

root@colibri-imx7:~# zcat /proc/config.gz | grep -i "trace"
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
# CONFIG_TREE_RCU_TRACE is not set
CONFIG_TRACEPOINTS=y
CONFIG_HAVE_ARCH_TRACEHOOK=y
# CONFIG_FENCE_TRACE is not set
# CONFIG_TRACE_SINK is not set
CONFIG_TRACE_IRQFLAGS=y
CONFIG_STACKTRACE=y
# CONFIG_RCU_TRACE is not set
CONFIG_NOP_TRACER=y
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
CONFIG_TRACER_MAX_TRACE=y
CONFIG_TRACE_CLOCK=y
CONFIG_CONTEXT_SWITCH_TRACER=y
CONFIG_GENERIC_TRACER=y
CONFIG_FTRACE=y
CONFIG_FUNCTION_TRACER=y
CONFIG_FUNCTION_GRAPH_TRACER=y
CONFIG_IRQSOFF_TRACER=y
CONFIG_SCHED_TRACER=y
CONFIG_FTRACE_SYSCALLS=y
CONFIG_TRACER_SNAPSHOT=y
CONFIG_TRACER_SNAPSHOT_PER_CPU_SWAP=y
CONFIG_STACK_TRACER=y
CONFIG_BLK_DEV_IO_TRACE=y
CONFIG_DYNAMIC_FTRACE=y
CONFIG_FTRACE_MCOUNT_RECORD=y
# CONFIG_FTRACE_STARTUP_TEST is not set
# CONFIG_TRACEPOINT_BENCHMARK is not set
# CONFIG_TRACE_ENUM_MAP_FILE is not set
# CONFIG_BACKTRACE_SELF_TEST is not set
3

Is there a way to check the built package before actually putting the image on an SD card?

4

You can enable buildhistory in your conf/local.conf to get more build information:

INHERIT += "buildhistory"

With that you get files such as

buildhistory/images/colibri_imx7/glibc/LXDE-image/installed-packages.txt
buildhistory/images/colibri_imx7/glibc/LXDE-image/installed-package-sizes.txt
5

I have been unable to replicate this. The built image never has ftrace enabled. I only end up with the following in /proc/config.gz:

root@colibri-imx7:~# zcat /proc/config.gz | grep -i "trace"
CONFIG_STACKTRACE_SUPPORT=y
CONFIG_TRACE_IRQFLAGS_SUPPORT=y
# CONFIG_TREE_RCU_TRACE is not set
CONFIG_HAVE_ARCH_TRACEHOOK=y
# CONFIG_FENCE_TRACE is not set
# CONFIG_TRACE_SINK is not set
CONFIG_STACKTRACE=y
# CONFIG_RCU_TRACE is not set
CONFIG_HAVE_FUNCTION_TRACER=y
CONFIG_HAVE_FUNCTION_GRAPH_TRACER=y
CONFIG_HAVE_DYNAMIC_FTRACE=y
CONFIG_HAVE_FTRACE_MCOUNT_RECORD=y
CONFIG_HAVE_SYSCALL_TRACEPOINTS=y
# CONFIG_FTRACE is not set
# CONFIG_BACKTRACE_SELF_TEST is not set

I wrote a simple script to hopefully produce the same build process each time.

#!/bin/bash

# Needed to build the menu config
sudo apt-get install libncurses5-dev

rm -rf oe-core
mkdir oe-core
cd oe-core
repo init -u http://git.toradex.com/toradex-bsp-platform.git -b LinuxImageV2.7
repo sync
. export

# Replace the machine line
sed -i "s/\(MACHINE ?= \"colibri-t20\)\"/#\1/g" 


# Dump in the stuff that we want
cat <<EOT >> conf/local.conf
MACHINE ?= "colibri-imx7"
ACCEPT_FSL_EULA = "1"
INHIBIT_PACKAGE_STRIP = "1"
EXTRA_IMAGE_FEATURES = "debug-tweaks"
INHERIT += "buildhistory"

EOT

# Build everything
MACHINE=colibri-imx7 bitbake -c menuconfig virtual/kernel
MACHINE=colibri-imx7 bitbake -f  -c compile virtual/kernel
MACHINE=colibri-imx7 bitbake console-trdx-image
6

To be clear during the menuconfig step I select the tracing packages. And when I run

grep -i "trace" ./tmp-glibc/work/colibri_imx7-angstrom-linux-gnueabi/linux-toradex/4.1-2.0.x-r0/build/.config

I do get the correct kernel configurations.

7

Is there maybe a problem in the update procedure? Is the kernel properly written to the UBI volume (after run update).

8

How would I check this?

9

The Linux banner at the very beginning contains a date/time stamp when it was compiled, that gives you a hint whether you are really booting that latest kernel you just built. Use dmesg | head.