With that in mind would anyone hold against me if I did my best to cut those out whenever I can do so without damaging the integrity of the adventure?
Depends, I guess. When are Challenges called for according to THE RULES?
And in regard to your specific question regarding our journey: why a Skill Challenge at all? This potentially shortens the story by a significant amount. "Ah, three failures in three rolls. You're unable to locate your destination. Markelhay thanks you for your time and Fallcrest's population lynches you on the way out. The end."
Taking care not to start sounding repetitive: this smells just like the fear I have about D&D being too much governed by rules. So as to our journey: why not point us in the right direction, have us walk for hours/days and slap an encounter (from a randomized table for all I care) on us, see what happens? In the end we reach our destination (if we live) and the trip there will have been a natural and eventful element of the story.
Why set up complex schemes like Challenges when you can just ask for independent checks as the situation requires - which also nicely circumvents the problem with players maximizing potential by delaying/aiding/generally being cu...cumbers - and let the story evolve based on the outcome?
Alternatively you could:
) have multiple players make any given check and decided that at least N of them have to succeed
) require success by at least N points
) require failure by no more than N points
) have multiple players check and calculate the total number by which the group succeeds/fails
etc.
There's plenty of possibilities with a little creativity.