I agree with pulling the whole pump, I hate pulling pullies. Three bolts are accessible with a socket from underneath. The fourth won't come out because of the pressure hose underneath but leave that one for last and the pump will come out as you loosen the bolt.You can actually get the whole assembly off without pulling the pulley off if you dont have a puller/installer. It will take a little longer but you can stick a wrench through the pulley to loosen the bolts. You can get a ratchet and socket on a couple of the bolts. To answer your question though yes it has to come off.
I work on a rack so pulling the hose is no big deal. Funny thing on tight, the three bolts go on easy and thread by hand the fourth though feels cross threaded the whole way but still tightens.I used to hate pulling that 4th bolt until I figured out how to get it from the front! Before I would take the hose off, which is a real big pain if you leave the pulley on. Taking the pulley on/off thing isn't much fun either, they go on tight.