I had a very similar issue awhile back. My speedo would not work while driving at all, when I bump it in neutral it would jump up and read steady, sometimes I would have to blip the throttle for it to read when it was in neutral. As soon as I put it back in gear it would drop dead and fall back to 0.
Fix: I got a new speedo driven gear (blue plastic), pulled out the old from the housing in the tranny and replaced it . All is good now. At first glance the original gear looks fine but if I looked really close, I could see a very slight groove/wear on it. This caused the drive gear to skip or not make contact to the driven gear causing the speedo cable to not spin.
That might be the case with yours...