The only way around it is to build a twin turbo set up with smaller T-3 style turbos. Just remember to bring cubic dollars to do it since you will need to fabricate new headers, intercooler, down pipes, then the expense of the turbo's.
Basically, you would be starting from scratch. If you want a turbo set up with a stick, you would be better off building a small block Chevy with twin turbos. I am not being cynical here. I, just like everyone else on here doesnt want to see you make a terrible mistake.
The Buick turbo V6 was and still is, a very well thought out combination of parts as it came from GM. From details like intake, header, camshaft, intercooler, transmission ratios and torque convertor stall. This combo of parts has put many cars into the high elevens on their stock parts. My own project car, has a Saginaw 4 speed and that is only because I had no other options as far as transaxles. I just wanted to Buick V6 since it fit so well into the car. Between the stick shift related parts and the transaxle, I could have bought 2 bullet proof TH2004r's from Bruce Toelle.