That is only looking at it one way. Without the rule, ball movement could suffer - more one on one type plays, rather than move the ball. Without the rule, there is no consequence. Everyone knew of the 5 second closely guarded rule and avoided it, which is why it is rarely called.
Heck, if you take your point, do away with the 5 second inbound, 10 second to cross half court and 3 second rules as well. They are seldom called either.