I don't recall telling you to raise the needle to #4. That would make it richer and you don't want or need that. You want to make it leaner, hence lowering the needle by moving the clip to slot #2.
The 3 circuits in your carbs overlap a little so changing one affects the one next to it. In the case of the midrange, it will affect both the lower part of the main and the upper part of the idle circuits .....
View attachment 349604
As you increase the main jet size, eventually the "bleed-over" into the upper midrange will make it too rich and cause stumbles and break-up under heavy or full throttle, say in the 4 to 5K range. To fix this you need to lean the needles a step (raise the clip to lower the needle). But changing the needle setting also affects the upper idle and the idle to midrange transition area, say from 2500 to 3500 RPMs. It can often create a flat spot there so to "fix" that, you increase the pilot jet one size.
So, I'd get 132.5 mains, keep the 27.5 pilots you have, and run 4M1 needles leaned one step (slot #2) from stock (slot #3).