So have I Dobro.

Most times without problems.
But to someone who has to ask, I prefer to be on the safe side - prevents recriminations and future posts like how do I get Bulletproof Black off my pants and desk top.
Certainly it's possible - with care and some precautions, but the nib will be inked (so fingers will probably get stained unless rubber gloves are worn), and dropping the body while fiddling with the nibs (been there, done that, cleaned up the mess) generally spatters ink out of the sac.
It also depends on how full the pen is. A just filled pen is going to prove most iffy.
Propping the nib-less barrel in something like a glass, if it needs to be set down while you hunt for the tissue you need to get the ink off your fingers, or to find the replacement nib that was just there a moment ago but rolled out of sight, is a good plan. Why not have a glass handy?
So, considering the above, I still recommend emptying the pen (but agree you can exchange nibs without emptying it), and rinsing is optional. If you're going to change colours - I'd rinse first, if not, don't bother. It just keeps the old ink colour out of the new nib is all.
Hope that helps.
Gerry