OSPF: Why the "Shortest Path" Isn't Always the Best Path

DC Consultant | DCACI | CCNP Security | PCNSA | CDE PAM
This week, I was deep in a multi-area OSPF lab on Cisco NX-OS and stumbled onto a scenario that defies standard "Cost" logic.
The lab setup:

4x Cisco N9K switches in different OSPF areas.
ASBR R4: Injecting a default route (0.0.0.0/0), Type E1, Metric 10. (Path via Backbone).
ASBR R7: Injecting a default route (0.0.0.0/0), Type E1, Metric 20. (Path via Area 2).
Router R5 (The ABR): Sits in the Backbone AND Area 2.
Internal Cost: To reach either ASBR, the internal cost is the same (40).
The Mystery:
Logic suggests R5 should prefer R4. Why? Because the total cost is lower:
Path R4: 10 (Ext) + 40 (Int) = 50
Path R7: 20 (Ext) + 40 (Int) = 60
But 'show ip route' on R5 indicated the winner was R7—the path with the higher cost.
The "Why": RFC 2328 Section 16.4.1
NX-OS strictly follows the RFC 2328 path preference hierarchy. Before OSPF even looks at the cost, it looks at the Area Type:
"Intra-area paths that use non-backbone areas are always the most preferred. The other paths, intra-area backbone paths and inter-area paths, are of equal preference."
Because R7 is in the same non-backbone area as R5, it is an Intra-area path. R4 is across the Backbone, also Intra-area with R5. But, as stated above, the non-backbone area is always preferred, so the path via R4 is less preferred.
The Twist: E1 vs. E2
Next, I’ve switched both external routes back to default Type E2. Suddenly, the logic flipped! R5 now preferres R4 (the lower metric).
Why? Because for Type 2 routes, OSPF follows Section 16.4 first:
"When all paths are type 2 external paths, the paths with the smallest advertised type 2 metric are always preferred."
In E2 selection, the Advertised Metric is a primary filter that happens before the Area Preference tie-breaker.
Takeaway:
The moral of the story? Modern NX-OS defaults to RFC 2328 logic (no compatibility rfc1583). In the old days of OSPF (RFC 1583), the router would have simply picked the lowest cost (R4) and moved on.
The most important lesson from this lab is that OSPF applies the RFC 2328 hierarchy at different stages depending on your Metric Type:
For Type-1 (E1): OSPF is Area-First. It prioritizes the "Gold Bucket" (Intra-area non-backbone) immediately. Because the Area Hierarchy is checked before the total cost, a non-backbone path wins regardless of how high the metric is.
For Type-2 (E2): OSPF is Metric-First. The advertised external metric (10 vs 20) is a hard filter. The "Area Awareness" only kicks in as a tie-breaker if those external metrics are identical.


