Advanced build system tricks {#advanced-build-system-tricks} ============================ [TOC] Introduction {#introduction} ============ This page describes some build systems tricks that can help developers but are not part of the standard workflow. They are low level commands that should not be taken as part of a stable API but better have a documentation than only having a description in the build system code. Customize the build system {#customize-build-system} ========================== + `RIOT_MAKEFILES_GLOBAL_PRE`: files parsed before the body of `$RIOTBASE/Makefile.include` + `RIOT_MAKEFILES_GLOBAL_POST`: files parsed after the body of `$RIOTBASE/Makefile.include` The variables are a list of files that will be included by `$RIOTBASE/Makefile.include`. They will be handled as relative to the application directory if the path is relative. Usage ----- You can configure your own files that will be parsed by the build system main `Makefile.include` file before or after its main body, examples usages can be: * Globally overwrite a variable, like `TERMPROG` * Specify a hard written `PORT` / `DEBUG_ADAPTER_ID` for some BOARD values * Define your custom targets * Override default targets Handling multiple boards with udev-rules {#multiple-boards-udev} ======================================== When developing and working with multiple boards the default `PORT` configuration for a particular board might not apply anymore so `PORT` will need to be specified whenever calling `make term/test`. This can also happen if multiple `DEBUGGERS/PROGRAMMERS` are present so `DEBUG_ADAPTER_ID` will also need to be passed. Keeping track of this will become annoying. One way of handling this is to use `udev` rules to define `SYMLINKS` between the boards serial port (`riot/tty-`) and the actual serial port (dev/ttyACM* or other). With this we can query the rest of the boards serial `dev` information (`DEBUG_ADAPTER_ID`, `PORT`, etc.) to always flash and open a terminal on the correct port. Procedure: - use `udevadm info /dev/ttyACM0` to query the udev database for information on device on port `/dev/ttyACM0`. or use `udevadm info --attribute-walk --name /dev/ttyACM0` for more detailed output when the first level of information isn't enough - create a udev rule with information of the device and one parent to create a matching rule in `/etc/udev/rules.d/70-riotboards.rules`. ~~~~~~~~~~~~~~~~~~~ # samr21-xpro SUBSYSTEM=="tty", SUBSYSTEMS=="usb", ATTRS{idVendor}=="03eb", \ ATTRS{idProduct}=="2111", ATTRS{manufacturer}=="Atmel Corp.", \ ATTRS{serial}=="ATML2127031800004957", SYMLINK+="riot/tty-samr21-xpro" ~~~~~~~~~~~~~~~~~~~ - reload rules: `udevadm control --reload-rules` - Boards `PORT` are symlinked to /dev/riot/tty-`board-name`. - Create a `makefile.pre` that will query the real `PORT` and the `DEBUG_ADAPTER_ID` from the `SYMLINK` info ~~~~~~~~~~~~~~~~~~~ PORT = /dev/riot/tty-$(BOARD) DEBUG_ADAPTER_ID = $(\ shell udevadm info -q property $(PORT) |\ sed -n ’/ID_SERIAL_SHORT/ {s/ID_SERIAL_SHORT=//p}’) ~~~~~~~~~~~~~~~~~~~ - You can now add `makefile.pre` to `RIOT_MAKEFILES_GLOBAL_PRE` as an environment variable or on each `make` call: ~~~~~~~~~~~~~~~~~~~ $ RIOT_MAKEFILES_GLOBAL_PRE=/path/to/makefile.pre make -C examples/hello-world flash term ~~~~~~~~~~~~~~~~~~~ _note_: if set as an environment variable it would be a good idea to add a variable to enable/disable it, e.g: ~~~~~~~~~~~~~~~~~~~ ifeq (1,$(ENABLE_LOCAL_BOARDS)) PORT = /dev/riot/tty-$(BOARD) DEBUG_ADAPTER_ID = $(\ shell udevadm info -q property $(PORT) |\ sed -n ’/ID_SERIAL_SHORT/ {s/ID_SERIAL_SHORT=//p}’) endif ~~~~~~~~~~~~~~~~~~~ Handling multiple versions of the same BOARD ------------------------------------------- The above procedure works fine when handling different boards, but not multiple times the same board, e.g: multiple `samr21-xpro`. An option for this would be to add an identifier of that board to the mapped `riot/tty-*`, there are multiple ways of handling this but in the end it means having a way to identify every copy. Another way would be to map the `DEBUG_ADAPTER_ID` in the name: ~~~~~~~~~~~~~~~~~~~ SYMLINK+="riot/node-$attr{serial} ~~~~~~~~~~~~~~~~~~~ But it will require to know in advance the serial number of each board you want to use. Another option would be to add some kind of numbering and defining multiple symlinks for each board. e.g. for `samr21-xpro` number `n`: ~~~~~~~~~~~~~~~~~~~ # samr21-xpro SUBSYSTEM=="tty", SUBSYSTEMS=="usb", ATTRS{idVendor}=="03eb", \ ATTRS{idProduct}=="2111", ATTRS{manufacturer}=="Atmel Corp.", \ ATTRS{serial}=="ATML2127031800004957", SYMLINK+="riot/tty-samr21-xpro", \ SYMLINK+="riot/tty-samr21-xpro-n" ~~~~~~~~~~~~~~~~~~~ Then, when flashing, the number can be specified and the parsing adapted: ~~~~~~~~~~~~~~~~~~~ ifneq(,$(BOARD_NUM)) PORT = /dev/riot/tty-$(BOARD)-$(BOARD_NUM) else PORT = /dev/riot/tty-$(BOARD) endif DEBUG_ADAPTER_ID = $(\ shell udevadm info -q property $(PORT) |\ sed -n ’/ID_SERIAL_SHORT/ {s/ID_SERIAL_SHORT=//p}’) ~~~~~~~~~~~~~~~~~~~ ~~~~~~~~~~~~~~~~~~~ BOARD=samr21-xpro BOARD_NUM=n make flash term ~~~~~~~~~~~~~~~~~~~ In the end, this would be the same as using the serial, but a simple number might be easier to handle. Notes ----- Udev only parses SUBSYSTEM and one parent. For others, we will rely on ENV variables defined by 60-serial.rules So the current filename should be higher than 60-serial.rules If for some reason re-writing the serial is needed there is a windows tool: https://remoteqth.com/wiki/index.php?page=How+to+set+usb+device+SerialNumber Documentation: -------------- * The whole documentation http://reactivated.net/writing_udev_rules.html#udevinfo * Udev manpage http://manpages.ubuntu.com/manpages/eoan/en/man7/udev.7.html # Handling multiple boards without udev-rules ============================================= This is a simpler approach to the above mentioned issue. The solution here only uses a makefile script for selecting the debugger and serial port. No administrative privileges (e.g. to configure Udev) are required. One of the limitations of the solution described here is that it currently doesn't work with multiple boards of the same type. This limitation is a limitation of the script and not of the mechanism used, it is possible to adapt the script to support multiple boards of the same type. This modification is left as an exercise to the reader. The following Make snippet is used: ~~~~~~~~~~~~~~~~~~~ LOCAL_BOARD_MAP ?= 1 # Adapt this list to your board collection SERIAL_nucleo-f103rb ?= 066BFF343633464257254156 SERIAL_same54-xpro ?= ATML2748051800005053 SERIAL_samr21-xpro ?= ATML2127031800008360 SERIAL_nrf52dk ?= 000682223007 ifeq (1,$(LOCAL_BOARD_MAP)) # Retrieve the serial of the selected board BOARD_SERIAL = $(SERIAL_$(BOARD)) # Check if there is a serial for the board ifneq (,$(BOARD_SERIAL)) # Set the variables used by various debug tools to the selected serial SERIAL ?= $(BOARD_SERIAL) DEBUG_ADAPTER_ID ?= $(BOARD_SERIAL) JLINK_SERIAL ?= $(BOARD_SERIAL) # Use the existing script to grab the matching /dev/ttyACM* device PORT_LINUX ?= $(firstword $(shell $(RIOTTOOLS)/usb-serial/find-tty.sh $(SERIAL))) endif endif ~~~~~~~~~~~~~~~~~~~ The array of board serial numbers has to be edited to match your local boards. The serial numbers used here is the USB device serial number as reported by the debugger hardware. With the `make list-ttys` it is reported as the 'serial': ~~~~~~~~~~~~~~~~~~~ $ make list-ttys /sys/bus/usb/devices/1-1.4.4: Atmel Corp. EDBG CMSIS-DAP, serial: 'ATML2127031800008360', tty(s): ttyACM1 /sys/bus/usb/devices/1-1.4.3: SEGGER J-Link, serial: '000683806234', tty(s): ttyACM0 ~~~~~~~~~~~~~~~~~~~ When the above make snippet is included as `RIOT_MAKEFILES_GLOBAL_PRE`, the serial number of the USB device is automatically set if the used board is included in the script. This will then ensure that the board debugger is used for flashing and the board serial device is used when starting the serial console. Analyze dependency resolution {#analyze-depedency-resolution} ============================= When refactoring dependency handling or modifying variables used for dependency resolution, one may want to evaluate the impact on the existing applications. This describe some debug targets to dump variables used during dependency resolution. To analyze one board and application run the following commands in an application directory. Generate the variables dump with the normal dependency resolution to a `dependencies_info_board_name` file: ~~~~~~~~~~~~~~~~~~~ BOARD=board_name make dependency-debug ~~~~~~~~~~~~~~~~~~~ Or with the "quick" version used by murdock to know supported boards (this is an incomplete resolution, details in `makefiles/dependencies_debug.inc.mk`) to a `dependencies_info-boards-supported_board_name` file: ~~~~~~~~~~~~~~~~~~~ BOARDS=board_name DEPENDENCY_DEBUG=1 make info-boards-supported ~~~~~~~~~~~~~~~~~~~ For more configuration and usage details, see in the file defining the targets `makefiles/dependencies_debug.inc.mk` To do a repository wide analysis, you can use the script `dist/tools/buildsystem_sanity_check/save_all_dependencies_resolution_variables.sh` that will generate the output for all boards and applications. It currently take around 2 hours on an 8 cores machine with ssd.