depends on what you mean by not holding ,
if after you turn the car off the pressure drops down slowly over about 1/2 hor or maybe more thats ok
if its within just a minute than something is wrong or you have a leak , doesnt take much of a leak with liquid to drop the pressure rapidly
you can crimp the rubber line at the front of the motor
once you have pressure ,crimp the feed hose if the pressure holds then the problem is not the regulator or injector(s) and there is probably is a leak in the feed line from pump to the engine , could be the rubber line from pump to the hange in the tank or the check valve in the pump isnt holding .or a rusted metal line by the frame
if it doesnt hold when you crimp the feed hose then it could be leaking injector(s) or fuel rail leak or the regulator
if it didnt happen before the swap look for a leak near the regulator oranything you may have touched in the fuel line
putting a known good regulator on would be a quick way to determine if its at fault , if it holds pressure , youve found the problem
but stating that the regulator shouldnt hold pressure is BS, i have the accufab on my 86 and it holds just fine