NULL Rates in Freight Data: Why They Matter More Than You Think

The Silent Corruption of Missing Data

Every freight forwarder has them. Fields left blank, values never entered, records half-complete. In a typical logistics dataset, NULL rates — the percentage of records where a critical field has no value — range from 3% to 20% depending on the field and the company. Most teams ignore them. After all, the system still works, shipments still move, and invoices still get paid. But the downstream consequences of missing data are far more severe than most logistics professionals realize.

When your analytics platform calculates average margin per shipment and 8% of records have no carrier cost, that average is wrong. Not slightly wrong — systematically biased. The missing records are not random; they tend to cluster around specific carriers, routes, or time periods. This means your "average" margin is actually the average of whichever shipments happened to have complete data, which may or may not represent your true performance.

The same logic applies to every metric you track. On-time delivery rates calculated on incomplete delivery dates. Revenue per customer calculated with missing invoice amounts. Carrier performance scores built on partial datasets. Each metric looks precise in the dashboard but carries a hidden margin of error proportional to the NULL rate of its underlying fields.

Measuring the Quality Behind Every KPI

Most logistics companies track operational KPIs — margin, OTD, cost per shipment — but almost none track the quality of the data behind those KPIs. This is like measuring the speed of a car without checking whether the speedometer is calibrated. NULL rates are the simplest and most powerful data quality metric you can implement, because they answer a fundamental question: how much of our data is actually usable?

A 10% NULL rate on carrier cost means your margin calculations are based on only 90% of shipments — and the missing 10% are rarely random. They represent systematic blind spots in your financial visibility.

Tracking NULL rates per field, per source system, per branch, and over time reveals patterns that aggregate completeness scores miss. You might discover that your ocean freight data has a 2% NULL rate on carrier cost while your road freight data has 18% — because the road carriers submit data differently. You might find that NULL rates spike every quarter-end when data entry teams are overloaded with volume.

Acceptable NULL Thresholds by Field Type

Not every field needs to be 100% complete. The acceptable threshold depends on how the field is used and when in the shipment lifecycle the value becomes available:

• Financial fields (carrier cost, shipper revenue): Target 99%+ completeness. Even 1% missing values distort margin calculations at scale. These fields should be mandatory at invoice stage.
• Date fields (ETD, ETA, actual delivery): Target 95%+ for completed shipments. Some incomplete records are expected for in-transit shipments, but any completed shipment should have full date coverage.
• Reference fields (booking ref, PO number): Target 90%+. Not all shipments have customer PO numbers, but internal references should be universal.
• Descriptive fields (commodity, notes): Target 70%+. These are useful for segmentation but not critical for financial or operational metrics.

How NULLs Corrupt Specific Analytics

Let us trace the impact of missing data through three common analytics scenarios to illustrate why NULL rates matter far more than most teams appreciate.

Margin analysis: Your dashboard shows an average margin of 14.2% across 5,000 shipments last month. But 400 of those shipments (8%) have no carrier cost. The system excluded them from the calculation. If those 400 shipments had an average margin of 6% — which is plausible since the records most likely to miss cost data are often the problematic ones — your true average margin is closer to 13.5%. That 0.7% difference, applied across annual revenue, could represent hundreds of thousands in misunderstood profitability.

Carrier scorecards: You rank carriers by on-time delivery rate. Carrier A shows 94% OTD based on 800 shipments. But 150 of their shipments have no delivery date recorded. If those 150 were disproportionately late — delayed shipments often have delayed data entry — Carrier A's true OTD might be closer to 87%. Your scorecard is telling you Carrier A is excellent when they may actually be underperforming.

Customer profitability: A customer appears highly profitable at €45 margin per shipment. But 12% of their shipments have missing revenue data. If those missing shipments were discounted or had special pricing, the true margin could be significantly lower.

Detection Methods: Finding Your NULL Hotspots

The first step is measurement. Run a completeness profile across every critical field in your dataset. For each field, calculate the NULL rate as a percentage of total records. Then segment the results by source system, branch, transport mode, and time period to identify patterns.

Syntask performs this profiling automatically on every data import, generating a completeness heatmap that highlights which fields, sources, and time periods have the highest NULL rates. This eliminates the need for manual SQL queries or spreadsheet analysis.

Look for three types of patterns: systematic NULLs (a specific source system never sends a particular field), temporal NULLs (completeness drops during certain periods), and random NULLs (scattered missing values with no clear pattern). Each type requires a different remediation approach.

Preventing NULLs at the Source, Not Backfilling Them

Fixing NULL rates requires addressing root causes, not just filling in blanks. Retroactively filling NULL values with averages or estimates introduces a different kind of error — you are replacing unknown values with assumptions, which can be worse than acknowledging the gap.

Instead, focus on prevention:

• Mandatory field validation: Configure your TMS to require critical fields before a record can be saved or a stage can be advanced.
• Integration mapping audits: Review every system integration to ensure all critical fields are mapped and flowing correctly.
• Process checkpoints: Add data completeness checks at key stages — booking, pickup, delivery, invoicing — so gaps are caught when the data is still fresh.
• Automated alerts: Set up notifications when NULL rates for any critical field exceed their threshold, enabling immediate investigation.

The goal is not perfection. It is awareness. When you know your carrier cost NULL rate is 4% and trending downward, you can confidently use your margin analytics while acknowledging a known, quantified limitation. When you do not track NULL rates at all, every analysis carries an unknown margin of error — which is far more dangerous.