Just a few ideas......
At some point, raising the boost increases the heat of the incoming charge to where the resulting temperature increase negates the addional pressure. A better intercooler would help here. The other possibility is that the engine has a bottleneck somewhere in that it can only accept 20 psi, the other 4 psi shown in the intake manifold simply doesn't make though the motor. Better heads and exhaust helps here. Could be a bunch of other things too.