Eventually you are going to get small bits of metal (parts of the berring) in the transmission fluid. Not sure about perminate damage but the valves in my trans stuck every once in a while. Since my TC has been replaced, theres been no problem.
I was told that if I let it go too long, the berring could go out, the TC would lock up and the trannie could be torn apart from the inside.
BOOSTD