With .472 intake and .496 on the exhaust I can guarantee that the retainers are hitting the guides. You won't see any damage to either, but I would bet money on it that they were hitting causing the noise. I would pull the rest of the lifters to inspect, then roll the push rods on a flat surface to be sure they are straight. Pull the heads and cut the guides for Viton seals. TA Performance has .500 and .531 Viton seals. You need to know the size of the cutter before ordering the seals. Various cam company's list cutters. They are sometimes hard to find. Hopefully the cam and lifters are all ok.