In this post I compare the apparent cyclomatic complexity and compiled size of two LoRaWAN implementations: LDL and LoRaMac-Node.

LDL is my own implementation. LoRaMac-Node appears to be a Semtech sponsored LoRaWAN reference implementation.

LoRaMac-Node has been around for a while and is probably a sensible choice for commercial projects. LDL is experimental and should not be trusted.

Method

The comparison is between LDL 0.2.3 and LoRaMac-Node 4.4.2.

Source to be compared is grouped into one of three domains:

Domain Definitions
Domain Description
Region region specific features
Radio radio drivers
MAC all platform agnostic code which does not fall into the region or radio domains, excluding cryptographic modes

The following source files are selected from each project for each domain:

Radio Domain Files
LDL LoRaMac-Node
ldl_radio.c/h radio/radio.h
ldl_radio_defs.h radio/sx1272/sx1272.c/h
ldl_chip.h radio/sx1276/sx1276.c/h
radio/sx1272/sx1272Regs-LoRa.h
adio/sx1272/sx1272Regs-Fsk.h
radio/sx1276/sx1276Regs-LoRa.h
radio/sx1276/sx1276Regs-Fsk.h
Region Domain Files
LDL LoRaMac-Node
ldl_region.c/h region/Region.c/h
region/RegionCommon.c/h
region/RegionEU868.c/h
region/RegionUS915.c/h
region/RegionEU433.c/h
region/RegionAU915.c/h
MAC Domain Files
LDL LoRaMac-Node
ldl_mac.c/h mac/LoRaMac.c/h
ldl_mac_commands.c/h mac/LoRaMacCommands.c/h
ldl_frame.c/h mac/LoRaMacConfirmQueue.c/h
ldl_ops.c/h mac/LoRaMacSerializer.c/h
ldl_sm.c/h mac/LoRaMacAdr.c/h
ldl_sm_internal.h mac/LoRaMacParser.c/h
ldl_stream.c/h mac/LoRaMacCrypto.c/h
ldl_system.c/h system/timer.c/h
peripherals/soft-se/soft-se.c/h
mac/LoRaMacHeaderTypes.h
mac/LoRaMacTypes.h
mac/secure-element.h
mac/LoRaMacTest.h

The comparison excludes:

  • cryptographic implementations (AES-ECB, AES-CMAC, AES-CTR)
  • target specific code (e.g. hardware abstraction layer)
  • mismatched LoRaWAN features which are implemented in separate files

LoRaMac-Node implements more regions, radios, and modes than LDL. Where possible these extra features have been excluded in order to create a more like for like comparison.

Unfortunately not all mismatched features can be excluded. For example, LoRaMac-Node class C is inseparable from class A, while LDL only implements class A. This difference means we should expect LoRaMac-Node to have more LoC and associated scores.

The table below gives a high-level view of the features implemented by the files sorted into the domain tables above. Mismatched features are indicated by a ‘yes’ in one column and a ‘no’ in the other.

Feature Matrix
Domain Feature LDL LoRaMac-Node
MAC Class A yes yes
MAC Class C no yes
MAC LoRaWAN 1.1 key derivation and handling yes no
MAC OTAA yes yes
MAC ABP no yes
MAC ADR yes yes
MAC Confirmed/Unconfirmed Data yes yes
MAC ioctl style access to all parameters no yes
Radio SX1272 yes yes
Radio SX1276 yes yes
Radio FSK Modulation no yes
Region EU_868_870 yes yes
Region EU_433 yes yes
Region US_902_928 yes yes
Region AU_915_928 yes yes

The tool pmccabe is used to measure:

  • apparent lines of code (includes comments and whitespace)
  • number of statements
  • number of functions
  • modified McCabe complexity (per function and per source file)

The higher the McCabe score, the more complicated the code. Functions with scores above 10 are considered complicated.

To determine compiled size, the source is compiled for an ARM Cortex M0+ using arm-none-aebi-gcc 8.2.1. All compiler settings are in the makefile. Size optimisation (-Os) is used in an effort to produce the smallest possible compiled size.

The objects are grouped into archive files and the size of each archive is determined using arm-none-aebi-size. The objects are not linked.

Both projects are vendored into this repository. The tools pmccabe and gcc are orchestrated from this script.

Data is always presented in the order of LDL first. LDL is plotted in blue, LoRaMac-Node is plotted in orange.

Overall Scores

LDL PMCCABE Results
DomainLoCFunctionsStatementsComplexity
Mac69331491812462
Radio12053628972
Region11272441883
92652092519617
LoRaMAC-Node PMCCABE Results
DomainLoCFunctionsStatementsComplexity
Mac137981893381862
Radio8129711358345
Region99151552347512
3184241570861719

Function Complexity

Here I’ve plotted the frequency of function complexity scores for each domain.

Compiled Size

LDL Compiled Size
Object File (from *.c)Text (bytes)BSS+Data (bytes)
ldl_region.o15000
ldl_radio.o14370
ldl_dummy_radio.o2016
ldl_mac.o79820
ldl_mac_commands.o7820
ldl_frame.o8880
ldl_ops.o13400
ldl_sm.o2620
ldl_stream.o5040
ldl_system.o180
ldl_dummy_mac.o36992
147691008
LoRaMAC-Node Compiled Size
Object File (from *.c)Text (bytes)BSS+Data (bytes)
Region.o13720
RegionAU915.o2810917
RegionCommon.o11840
RegionEU433.o2675212
RegionEU868.o2883292
RegionUS915.o3000920
sx1276.o6092284
sx1272.o5517284
LoRaMac.o121001644
LoRaMacCommands.o608256
LoRaMacConfirmQueue.o58042
LoRaMacSerializer.o4840
LoRaMacAdr.o2040
LoRaMacParser.o3340
LoRaMacCrypto.o229284
timer.o5304
soft-se.o1040968
437055907

There used to be some graphs here produced by linking the above with an empty demo app. I’ve removed it since the placement of the graph looked like it was using the data in the tables but it was actually using the output from toolchain size tool after the linker had done its thing.