It is possible that you could be hearing the shaft rubbing on the bearing, but I highly doubt it due to the film of oil on the bearings and shaft. What you are probably hearing is the compressor wheel rubbing on the compressor cover, or turbine wheel rubbing on the turbine housing. If you can pull the shaft far enough to the side that the wheel touches the housing or rubs on the housing when spun, the bearings are starting to go bad. I would recommend you have it at least looked at by a reputable turbo repair facility as a bare minimum. If the wheel is rubbing on the compressor cover, it's only a matter of time before the tiny pieces of aluminum turn into big chunks of compressor blade and compressor cover. These big chunks could possibly cause damage to your intercooler if lets say a piece of the blade breaks off under WOT and goes flying ballistically into your intercooler. Not too metion if it made it out of the intercooler and into your engine. No questions are stupid questions. We all have to learn how to walk before we can run.
Just my O2's worth.
Patrick