RIOT/doc/doxygen/src/advanced-build-system-tricks.md

Advanced build system tricks

[TOC]

Introduction

This page describes some build systems tricks that can help developers but are not part of the standard workflow.

They are low level commands that should not be taken as part of a stable API but better have a documentation than only having a description in the build system code.

Customize the build system

• RIOT_MAKEFILES_GLOBAL_PRE: files parsed before the body of $RIOTBASE/Makefile.include
• RIOT_MAKEFILES_GLOBAL_POST: files parsed after the body of $RIOTBASE/Makefile.include

The variables are a list of files that will be included by $RIOTBASE/Makefile.include. They will be handled as relative to the application directory if the path is relative.

Usage

You can configure your own files that will be parsed by the build system main Makefile.include file before or after its main body, examples usages can be:

• Globally overwrite a variable, like TERMPROG
• Specify a hard written PORT / DEBUG_ADAPTER_ID for some BOARD values
• Define your custom targets
• Override default targets

Handling multiple boards with udev-rules

When developing and working with multiple boards the default PORT configuration for a particular board might not apply anymore so PORT will need to be specified whenever calling make term/test. This can also happen if multiple DEBUGGERS/PROGRAMMERS are present so DEBUG_ADAPTER_ID will also need to be passed. Keeping track of this will become annoying.

One way of handling this is to use udev rules to define SYMLINKS between the boards serial port (riot/tty-<board-name>) and the actual serial port (dev/ttyACM* or other). With this we can query the rest of the boards serial dev information (DEBUG_ADAPTER_ID, PORT, etc.) to always flash and open a terminal on the correct port.

Procedure:

• use udevadm info /dev/ttyACM0 to query the udev database for information on device on port /dev/ttyACM0.

  or use udevadm info --attribute-walk --name /dev/ttyACM0 for more detailed output when the first level of information isn't enough

• create a udev rule with information of the device and one parent to create a matching rule in /etc/udev/rules.d/70-riotboards.rules.

    # samr21-xpro
    SUBSYSTEM=="tty", SUBSYSTEMS=="usb", ATTRS{idVendor}=="03eb", \
    ATTRS{idProduct}=="2111", ATTRS{manufacturer}=="Atmel Corp.", \
    ATTRS{serial}=="ATML2127031800004957", SYMLINK+="riot/tty-samr21-xpro"

• reload rules: udevadm control --reload-rules

• Boards PORT are symlinked to /dev/riot/tty-board-name.

• Create a makefile.pre that will query the real PORT and the DEBUG_ADAPTER_ID from the SYMLINK info

    PORT = /dev/riot/tty-$(BOARD)
    DEBUG_ADAPTER_ID = $(\
        shell udevadm info -q property $(PORT) |\
        sed -n ’/ID_SERIAL_SHORT/ {s/ID_SERIAL_SHORT=//p}’)

• You can now add makefile.pre to RIOT_MAKEFILES_GLOBAL_PRE as an environment variable or on each make call:

    $ RIOT_MAKEFILES_GLOBAL_PRE=/path/to/makefile.pre make -C examples/hello-world flash term

note: if set as an environment variable it would be a good idea to add a variable to enable/disable it, e.g:

ifeq (1,$(ENABLE_LOCAL_BOARDS))
    PORT = /dev/riot/tty-$(BOARD)
    DEBUG_ADAPTER_ID = $(\
        shell udevadm info -q property $(PORT) |\
        sed -n ’/ID_SERIAL_SHORT/ {s/ID_SERIAL_SHORT=//p}’)
endif

Handling multiple versions of the same BOARD

The above procedure works fine when handling different boards, but not multiple times the same board, e.g: multiple samr21-xpro.

An option for this would be to add an identifier of that board to the mapped riot/tty-*, there are multiple ways of handling this but in the end it means having a way to identify every copy.

Another way would be to map the DEBUG_ADAPTER_ID in the name:

SYMLINK+="riot/node-$attr{serial}

But it will require to know in advance the serial number of each board you want to use. Another option would be to add some kind of numbering and defining multiple symlinks for each board. e.g. for samr21-xpro number n:

    # samr21-xpro
    SUBSYSTEM=="tty", SUBSYSTEMS=="usb", ATTRS{idVendor}=="03eb", \
    ATTRS{idProduct}=="2111", ATTRS{manufacturer}=="Atmel Corp.", \
    ATTRS{serial}=="ATML2127031800004957", SYMLINK+="riot/tty-samr21-xpro", \
    SYMLINK+="riot/tty-samr21-xpro-n"

Then, when flashing, the number can be specified and the parsing adapted:

    ifneq(,$(BOARD_NUM))
      PORT = /dev/riot/tty-$(BOARD)-$(BOARD_NUM)
    else
      PORT = /dev/riot/tty-$(BOARD)
    endif
    DEBUG_ADAPTER_ID = $(\
      shell udevadm info -q property $(PORT) |\
      sed -n ’/ID_SERIAL_SHORT/ {s/ID_SERIAL_SHORT=//p}’)

    BOARD=samr21-xpro BOARD_NUM=n make flash term

In the end, this would be the same as using the serial, but a simple number might be easier to handle.

Notes

Udev only parses SUBSYSTEM and one parent. For others, we will rely on ENV variables defined by 60-serial.rules

So the current filename should be higher than 60-serial.rules

If for some reason re-writing the serial is needed there is a windows tool: https://remoteqth.com/wiki/index.php?page=How+to+set+usb+device+SerialNumber

Documentation:

• The whole documentation http://reactivated.net/writing_udev_rules.html#udevinfo
• Udev manpage http://manpages.ubuntu.com/manpages/eoan/en/man7/udev.7.html

Handling multiple boards without udev-rules

=============================================

This is a simpler approach to the above mentioned issue. The solution here only uses a makefile script for selecting the debugger and serial port. No administrative privileges (e.g. to configure Udev) are required.

One of the limitations of the solution described here is that it currently doesn't work with multiple boards of the same type. This limitation is a limitation of the script and not of the mechanism used, it is possible to adapt the script to support multiple boards of the same type. This modification is left as an exercise to the reader.

The following Make snippet is used:

    LOCAL_BOARD_MAP ?= 1

    # Adapt this list to your board collection
    SERIAL_nucleo-f103rb ?= 066BFF343633464257254156
    SERIAL_same54-xpro ?= ATML2748051800005053
    SERIAL_samr21-xpro ?= ATML2127031800008360
    SERIAL_nrf52dk ?= 000682223007

    ifeq (1,$(LOCAL_BOARD_MAP))

      # Retrieve the serial of the selected board
      BOARD_SERIAL = $(SERIAL_$(BOARD))

      # Check if there is a serial for the board
      ifneq (,$(BOARD_SERIAL))

        # Set the variables used by various debug tools to the selected serial
        SERIAL ?= $(BOARD_SERIAL)
        DEBUG_ADAPTER_ID ?= $(BOARD_SERIAL)
        JLINK_SERIAL ?= $(BOARD_SERIAL)

        # Use the existing script to grab the matching /dev/ttyACM* device
        PORT_LINUX ?= $(firstword $(shell $(RIOTTOOLS)/usb-serial/find-tty.sh $(SERIAL)))
      endif
    endif

The array of board serial numbers has to be edited to match your local boards. The serial numbers used here is the USB device serial number as reported by the debugger hardware. With the make list-ttys it is reported as the 'serial':

$ make list-ttys
/sys/bus/usb/devices/1-1.4.4: Atmel Corp. EDBG CMSIS-DAP, serial: 'ATML2127031800008360', tty(s): ttyACM1
/sys/bus/usb/devices/1-1.4.3: SEGGER J-Link, serial: '000683806234', tty(s): ttyACM0

When the above make snippet is included as RIOT_MAKEFILES_GLOBAL_PRE, the serial number of the USB device is automatically set if the used board is included in the script. This will then ensure that the board debugger is used for flashing and the board serial device is used when starting the serial console.

Analyze dependency resolution

When refactoring dependency handling or modifying variables used for dependency resolution, one may want to evaluate the impact on the existing applications. This describe some debug targets to dump variables used during dependency resolution.

To analyze one board and application run the following commands in an application directory.

Generate the variables dump with the normal dependency resolution to a dependencies_info_board_name file:

BOARD=board_name make dependency-debug

Or with the "quick" version used by murdock to know supported boards (this is an incomplete resolution, details in makefiles/dependencies_debug.inc.mk) to a dependencies_info-boards-supported_board_name file:

BOARDS=board_name DEPENDENCY_DEBUG=1 make info-boards-supported

For more configuration and usage details, see in the file defining the targets makefiles/dependencies_debug.inc.mk

To do a repository wide analysis, you can use the script dist/tools/buildsystem_sanity_check/save_all_dependencies_resolution_variables.sh that will generate the output for all boards and applications. It currently take around 2 hours on an 8 cores machine with ssd.