I think you and everybody else are making this more complicated than it really is.

The crux of your problem lies in how you started out your post:

"Having recently spent some time working hard at moving away from finger-picking the same pattern over and over again throughout a song"

Dude, repeating patterns are your friends.The idea that you're supposed to sing a song and consciously think about what notes to play is just stupid. Our brains don't work that well at multi-tasking.

What you want to do is get one repeating pattern so ingrained into your playing that it's part of who you are. Once that happens the in between stuff and interesting improvisational melody line stuff is drop-dead simple.

To understand how this works it might be easier to look at the problem from a different angle.

In frailing banjo (and this isn't too far off the path because American folk guitar developed out of early banjo styles) we work with a single basic picking pattern of a quarter note and two eighth notes.


D --0--0-----
B -----0-----
G -----0-----
D -----0-----
G -------0---


It's played in an all down-picking motion, and it gives us a basic rhythm of 1 2 & or "bump dit-ty".

The wild thing about frailing banjo is that this simplistic pattern is amazingly adaptable.

Just playing a hammer-on:

      h
D --0^2--2-----
B -------0-----
G -------0-----
D -------0-----
G ---------0---

changes the count to 1 & 2 &.

That's cool, but it gets really interesting when you start mixing in rests.

If we play the initial quarter note as a rest but play the hammer-on anyway:

      h
D --x^2--2-----
B -------0-----
G -------0-----
D -------0-----
G ---------0---

we still wind up with a count of 1 & 2 & but it feels different.

We haven't really changed anything. The picking pattern is still the same and the hammer-on is still dropping in at the same point, but because we played one part of the pattern as a rest the phrasing of the pattern changed.

For a really basic, bozo-banjo, example of this in action listen to the Minglewood Blues workshop here: http://howandtao.com/?p=87 I start out the workshop playing the basic "bump dit-ty" and over the space of the workshop mix in some rests to put things into more of a blues framework.

Moving back to the guitar that idea of taking a simple pattern and using a mix of simple techniques to alter the phrasing works even better.

If you take something like a thumb-brush pattern:


E -----0-0---
B -----0-----
G -----1-----
D -----2-----
A -----2-----
E --0--------


there are almost endless combinations of rests, hammer-on's and "stuff" that can be tossed into the mix without breaking out of the basic repeating pattern.

If you start looking at how scales & chords are interrelated on the fretboard you'll start discovering at any melody note can be picked up by moving a finger here or there in a chord form.

Try strumming this pattern:

   
3-3-2-0-1-0    3-3-2-0-3-0    3-3-2-0-1-0   3-3-2-0-1-1

3-3-2-0-1-3    3-3-2-0-1-1    3-3-2-0-1-0   3-3-2-0-3-0



Then put it into a roll of some sort - and then see if you can find the melody to Freight Train.

Fore more information on this, check out the guitar stuff at: http://howandtao.com/?page_id=20

Don't overcomplicate the guitar. Simplify. Simplify. Simplify.

-Patrick