A "dirty bomb" would typically use enriched uranium or plutonium with half lives of up to 90 years. This means that the dust from the explosion could continue to be dangerous for this time. Only very little dust needs to remain (around 11 in ten millionths) for danger to be present.
A dirty bomb used (say) in Westminster, Heathrow or at Clapham Junction would probably not take many lives but would have catastrophic and long lasting economic and strategic consequences.