When you say you don't bother to visit special levels unless they benefit your build... I'm tempted to reply that all special levels benefit any build, for obvious XP purposes.
That's just the thing; on UV at least, I've come to notice that the XP is
way high enough to skip some of the special levels and still level up at very good pace. In fact, if there's anything UV has taught me, it's that there's absolutely no reason to stretch out for XP (I'm sure this can depend on your build choice, since some master traits need more xp and are more important to get to early than others) if it seems like even a remotely poor idea.
As a more extreme example, though I don't post my mortems, my last and by far most successful victories were:
-- Special levels --------------------------------------------
Levels generated : 11
Levels visited : 5
Levels completed : 4
Ammochain - The non-completed level was The Vaults (when applicable, I don't see any reason not to enter it just to clear two vaults when I don't have the AM's staff on me, even if it takes away my YAAM). Got to the true evil, but ended up getting killed when he started randomly teleporting off-screen whenever I was shooting at him. First time that's ever happened to me. Was weird.
-- Special levels --------------------------------------------
Levels generated : 11
Levels visited : 5
Levels completed : 4
Army of the Dead - Non-completed level was The Vaults; same reason as the one before it. Full win.
-- Special levels --------------------------------------------
Levels generated : 11
Levels visited : 2
Levels completed : 2
Gunrunner - Would've been 3 levels visited, but Phobos Lab spawned. Nope. As soon as I got Gunrunner, I just started dashing down stairways. I could have explored looking for a nuke or nuclear weapon, but I kind of hate this master-trait(it was a themed run), so I just went straight down and finished it. Standard victory.
-- Special levels --------------------------------------------
Levels generated : 11
Levels visited : 2
Levels completed : 2
Masterless Melee build - This one was actually the easiest run of them all (would've been 3, but, yeah, Phobos Lab spawned). Once again no nuke or nuclear weapon (I really hate how not only is the Spider's Lair paired, but it no longer has the nuclear plasma rifle), but no stress. Standard victory.
I know that these are just isolated examples, but it's funny just how successful and easy these runs were in comparison to ones where I attempted to visit a bunch of special levels. Admittedly, I got lucky and found a Super Shotgun in HA on my Army of the Dead build, but on every other one I actually had pretty eh luck and a decent batch of awful spawns.
So, while I understand why one would want to complete more special levels (otherwise I wouldn't have made this thread regarding a level that I genuinely can't see a decent tactical reason to visit without a homing phase), I don't see a reason to
do a good share of the levels unless it, obviously with a few exceptions, directly benefits your build with more than just XP.
Note that I don't play on N!. Not yet, anyways.