ZZBX / NOTES ← All notes
AutoSAR · Agent Engineering · Python

105 errors. 99 of them
were not real.

Building an ARXML × DBC consistency checker for an AI agent. The first version reported 105 errors on a real project. After verifying every one, 99 were false — the matrix describes eleven nodes, and most frames had nothing to do with this ECU. A false report is worse than a missing one: nobody reads past the hundredth line.

2026-11-05 ◆ 12 min read ◆ Python / stdlib ◆ Open source
26check codes
0dependencies
3real projects
0.31son 5.6 MB

Why nobody checks this layer

An AUTOSAR communication stack is configured in two artefacts maintained by two different groups: the ECU configuration ARXML the supplier delivers (5.6 MB, 23 modules, tens of thousands of ECUC containers), and the DBC communication matrix. Frame ids, lengths, every signal's start bit, byte order, init value and direction must agree between them.

When they disagree, the compiler says nothing. The most common failure I have seen: a start bit is off by three, so the signal is decoded from its neighbour's bits. Both sides run happily, the frame parses, and that one value is permanently wrong. You cannot see it in any log, because it looks like perfectly normal data.

Format What a mistake does
JSONParsing fails, with a line number
XMLThere is a schema
YAMLBad indentation is an error
ARXML / DBCEvery tool opens it. Frames keep being sent. The values are wrong.

So the first move was to look for an existing tool. There is none: canmatrix reads system-description ARXML (CAN-CLUSTER / FRAME) and never touches ECUC configuration; cogu/autosar is a parser with no validation; the local AUTOSAR lint has Com rules but no DBC cross-check. Nobody does this, because ECUC structure differs per vendor and a general parser is unglamorous work.

Real check output on a production project
One command: frame counts, ECU nodes, bus coverage, and per-finding detail
Constraints

Single file, standard library only, read-only. The last one matters more than it looks: a tool that can modify an ECU configuration will eventually modify one by accident.

Bit geometry: I had the direction wrong

This is the part worth telling in full. AUTOSAR stores a signal's start as ComBitPosition (or START-POSITION); a DBC stores it as the first number after SG_. They look like the same quantity, so you would just compare them. You cannot.

My first version assumed two conventions existed in the wild (MSB-first and LSB-first), computed both, and accepted a match from either. It felt robust: be lenient, get fewer false positives. Then I measured it on a real file.

Formula Hits
Sawtooth S+L-1 / flat [S, S+L)153 / 214(71.5%)
Sawtooth S / flat [S-L+1, S]33 / 214(15.4%)

Neither is good, which means the assumption itself is wrong. So I grouped by PDU to see whether some PDUs used one convention and others the other. Scene_Sync was 100% sawtooth; Node_Info and Time_Info were 100% flat. It genuinely looked like two schools.

Then I noticed the mistake: one of those PDUs uses little-endian signals and the other uses big-endian. I was reading an endianness difference as a convention difference.

What is actually true

AUTOSAR's ComBitPosition always stores the signal's LSB, in flat numbering (bit0 = LSB of byte0), independent of byte order. A DBC big-endian signal uses sawtooth numbering and writes the MSB as its start. So there is exactly one conversion: back off len-1 steps to find the MSB, then walk the sawtooth.

After the fix: 215 out of 215, zero failures. And the earlier 'two schools' observation dissolved — it was one formula with its direction reversed.

The decision I have to criticise

'Try both, accept either match' was wrong for two reasons. It launders real defects — when a start bit is genuinely wrong, the other interpretation sometimes matches by coincidence. And it rests on a false premise: the two schools did not exist. A lenient validator is not a good validator; it makes you feel good at the cost of letting real problems through.

99 false errors out of 105

First real project: 0 error, 16 warn. Second one, different supplier, different MCU: 5 errors, all genuine. Third one was a whole-network matrix — 91 frames, 11 nodes. It reported 105 errors.

My first reaction was that the configuration was a disaster. Then I grouped the 'frame has no CanIf PDU' findings by sender:

NodeA     22 frames
NodeB      7 frames
NodeC      6 frames
NodeD      4 frames
NodeE      2 frames
...
ECU_R      1 frame   ← the only one this ECU actually sends

Most of the 91 frames are one node talking to a third. My code assumed every frame in the DBC must be configured in this ECU's CanIf and Com. That assumption holds only for a single-ECU DBC — a dozen frames, BU_: listing mostly this ECU. The first two projects were exactly that shape, which is why this never surfaced. On a whole-network matrix it is simply false: for a frame this ECU is not involved in, missing configuration is the correct state.

The fix: decide per frame whether this ECU participates — the frame's sender is this ECU, or any of its signals lists this ECU as a receiver. If neither, skip it silently. Result 105 → 61, and the six genuine defects finally became visible.

Errors tallied by check code
After the fix: 61 errors. Before it: 105, of which 99 were false.
The bug behind the fix

That participation test relies on each signal's receiver list — the tail of an SG_ line: "" ECU_L,ECU_R. My code did split(), which splits on whitespace. The DBC spec uses commas. So the result was ['ECU_L,ECU_R'] — one element. The participation test had been dead on arrival: it always answered 'not a receiver', so every frame took the 'unrelated' branch.

Had I not checked that one line, the whole fix would have been cosmetic — 105 rewritten to 61 without any of it being true. Any fix that 'looks like it worked' still needs its input verified separately.

Skipping looks exactly like passing

This one hides inside success. I fed the tool an empty DBC to check robustness, and got: 0 error, 0 warn, 53 info. Green.

summary: 0 error, 0 warn, 53 info

The chain: zero frames → every bus hits 0% → all judged 'outside this DBC's coverage' → skipped wholesale → nothing was compared, and the report is clean. Hand a supplier the wrong file and you get a green report back.

Bad input must fail, not pass quietly
AXD-023: if nothing was compared, it must fail

New rule AXD-023 fails hard in both cases: a DBC with no frames, and no PDU landing inside the DBC's coverage. Skipping and passing must look completely different in a report. This is the same disease as section 02's 'try both conventions' — both substitute 'looks fine' for 'was checked'.

Six bugs only real files expose

Synthetic demo data passed every time. Real files broke six things at once.

Bug Consequence
Parameter tag is -PARAM-VALUE, I assumed -PARAMETER-VALUE Every CAN id / DLC / length read as None — while probe still printed container counts, so it looked like 'the file has no configuration'
ComSignal names carry an _IPDU_ suffix 226 false 'missing signal' + 252 false 'extra signal'
Matching Com ↔ CanIf by name The real topology does not work that way
Single-node direction check One MCU drives both sides; the far side is judged as a direction error
Bit-geometry formula reversed See section 02
TX PDU DLC read from CanIfTxPduDlc This vendor does not configure CanIfTxPduDlc at all; must come from EcuC PduLength

The name-matching one deserves emphasis. In a real configuration Com and CanIf are not directly connected — the chain runs frame → CanIf PDU → EcuC Pdu → PduR route → EcuC Pdu(COM) → Com IPdu. IPDU_COM_TP_Param and the frame Diag_CalData share no name similarity, yet they are the same link via COM_TP_Param → CanTp_TP_Param → 0x9C.

How to prove 0 error is real

A tool reporting 0 errors can mean two things: there are no problems, or the thresholds are so loose nothing can trip them. The report cannot tell you which. So you inject known defects into a real file and check that they surface.

[E] AXD-003  TX_PDU_ECU_L_St DLC disagrees with DBC: CanIf=8B / DBC=32B
[E] AXD-008  IPDU_COM_Node_Info.CarMode bit geometry mismatch

Both caught. That is what makes a clean run meaningful. The same logic went into the self-test: it asserts not only which codes fired but how many findings per severity — because my first version asserted only the code set, and when the fixture drifted the self-test still printed 'pass'.

Pattern

Three times in this project a test printed success while checking less than I thought: the code-set-only assertion, the whitespace split, and the empty DBC. All three are the same shape.

Letting a fresh session use it

Writing the tool and writing the docs are not the same as the agent being able to use it. So I installed the skill, started a brand-new session, and gave it two file paths and one sentence — no hints — to see whether it would find the skill, read SKILL.md, follow the order, call the installed copy rather than my repo, and report the version hash.

It did all of it correctly, with output matching my own run. More useful: it found four problems in my documentation.

Problem Nature
Docs said 'expect 8 error / 3 warn', actual was 13/2/1 Substantive: the self-test never asserted counts
Docs promised to report bus coverage percentages, but the code emitted them nowhere Substantive: promised something that did not exist
The example used --node LDM, left over from another project Misleading
No note that an empty 'notes' field is normal Missing documentation

The second one is the most instructive: SKILL.md instructed the agent to quote a bus-coverage line that the code never printed. Anything written in a skill that the code does not actually do is more dangerous than a bug, because the agent will faithfully follow it and produce a confident, wrong report.

My own mistake

The AXD-016 message read 'CanIf did not configure FD'. A user looked at the data and said: those frames are wrong in the DBC, not missing from the ARXML.

He was right, and the evidence is unambiguous: four frames configured as classic CAN with a 32-byte payload, and their signals genuinely fill all 32 bytes. Classic CAN tops out at 8. So FD is a fact — what is questionable is whether those frames belong in this ECU's matrix at all. My wording pointed straight at the ARXML, which would have sent someone to add an FD flag that fixes nothing.

Now the message computes the actual bit span from the Com IPdu and says something different depending on what it finds: if the signals really span more than 8 bytes, FD is a fact and the flag is wrong; if the frame should not be here at all, adding a flag will not help. Same code, different pointing, driven by evidence rather than a fixed assumption.

And an operational slip

While syncing that change I ran cp with the arguments reversed, overwriting the code change with the old copy — docs updated, code not. Only a diff -r catches that kind of thing.

Results and method

26 check codes across Com / PduR / CanIf / CanDrv. One file, standard library, about 2500 lines.

Project Scale Result
A21 frames, both sides0 error, 17 warn, 47 info
B25 frames, another MCU5 error, 4 warn, 13 info
Whole network91 frames, 11 nodes61 error, 15 warn, 76 info

5.6 MB ARXML in 0.31 seconds, 62 MB peak memory. Verification: self-test (codes and counts), contract test (documented codes match implemented codes in both Chinese and English docs), fault injection, and an isolated run with a cleared environment and no repository present.

Five things worth keeping

1. A lenient validator is not a good validator — 'try both, accept either' lets real defects through on coincidence.
2. Skipping and passing must look different in a report.
3. Synthetic fixtures must use the real topology, or the self-test validates a path that does not exist.
4. Every fix that 'looks like it worked' still needs its input verified separately.
5. Let a fresh session use the skill — it finds the places where the docs promise what the code does not do.

Common questions

Why do the ARXML and the DBC disagree when both come from the same project?

Because they are written by two different groups at two different times. The supplier delivers the ECU configuration ARXML; the OEM publishes the vehicle DBC matrix. Neither is generated from the other, and no build step compares them — so a change on one side simply does not propagate.

How do you compare an AUTOSAR bit position with a DBC start bit?

There is exactly one correct formula. AUTOSAR ComBitPosition stores the signal LSB, flat-numbered, regardless of byte order. DBC numbers Motorola signals in sawtooth form and stores the MSB. Walk back length-1 steps from the LSB to get the DBC start bit. Accepting 'either convention' is wrong: a genuine error gets whitewashed by a coincidental match.

Why did the first version report 105 errors when only 6 were real?

The DBC described a whole 11-node network, and the checker assumed every frame in it must have a configuration in this ECU. Most frames belong to other nodes, so they produced errors that were never actionable. Deciding per frame — 'is this ECU the sender, or does any signal's receiver include it?' — cut 99 false errors.

How do you prove a clean run is real and not a skipped check?

Inject known defects and confirm they are still caught, and make 'skipped' look different from 'passed' in the report. An empty DBC used to report 0 errors — zero frames meant every bus fell below the coverage threshold, so everything was judged out of scope and skipped. A clean report on a file that was never read is worse than a noisy one.

Harry Zhao
AUTOMOTIVE EMBEDDED · AUTOSAR BSW / PLATFORM TOOLING

12 years in automotive embedded software, from ECU low-level drivers up to AutoSAR CP platform architecture: build flows, test scripting, Git servers, Jenkins CI/CD, and chip-level functional safety on Infineon TC397 and NXP S32K314. The tools in this article came out of real supplier configuration reviews.

Read-only tool. All example data is synthetic; no real vehicle, supplier or customer identifiers appear.