It's worth taking a 'silent' bit of soundtrack (background noise only) and inserting it in the gap.

It's only three simple steps:

1) Take the audio event/clip out of it's grouping with the video by clicking once on the audio portion you want to deal with and hit 'U' to UNGROUP it
2) SPLIT the event just before and after the part you want to take out by clicking where you want to cut it and hit 'S' to SPLIT it, both before and after the fumble.
3) Delete the little piece you have now cut out (the 'fumble') by right-clicking once on the portion and choosing DELETE

If you ever cut out a portion you want back, just drag the edge of any of the events that are still there, the entire thing still 'exists in the background', it's just showing the edges of where your cuts are beginning and ending.

One good suggestion, depending on the audio in your compilation, is to 'hide' the edits you make. Most people use music for their home videos, so this is not needed as it hides small changes in talking well. If you are just talking off and on, as a commentary, then cutting and splicing your voice is fine. But, if you are taking a nice long panoramic view of your travels, or want a nice montage video with no sharp changes that seem abrupt to the viewer, then I suggest following the above suggestion and putting in some 'background' noise over the edit, which will 'hide' it a lot more.

That's a good idea for any project, if you have the time: adding a background track. Music is usually used for home users, but a nice ambient crowd, recorded in a similar setting, looping in the background (fade the ends!) makes for nice, solid footage.

Good luck with it and have fun!

Edit: Don't worry about leaving gaps (especially if it is just you talking). Many people correct their commentaries on YouTube/Vimeo/other sharing sites this way. Wasting a lot of time with audio envelopes, dipping your voice down (but not all the way) only muffles the fumble and it is still there for the world to hear, especially for those who have good ears! I suggest a background track over simply lowering the volume for that reason.

Leaving gaps in an audio track is both noticeable and annoying, and is usually not suggested. If one is able to blend or replace the snippet with ambient sound as TOG suggested above, that is preferable. When one has dozens of audio tracks and a short turnaround, as I often do, just dipping the glitches down near the audio floor is the most expedient, and usually is not noticeable.

+1 Musicvid - In audio edits there is usually nothing more glaring than slicing it down the middle of some audio track that is present and butting the ends together. Sounds like audio hacking and it does not sound good to the listener. Roll-offs and dips preferred.