mandolin has been part of celtic music since early 19th c., banjo and guitar came into it much later; you can read Fintan Valeley's compendium mandolin entry for some history; pick the fiddle tunes as fast as you can for jigs and reels, play some two or three note chords along the melody line, add some tasteful tremolo if it's a slower air; listen to early Andy Irvine, Paul Brady, Sweeney's Men, Planxty, the Johnstons, etc.; Mick Moloney has a nice mando/banjo cd called Strings Attached;

I'm struggling with this too, best thing I can think of to do is listen as much as you can, learn some more typically celtic chords rather than the bluegrass chop chords, celtic music tends (not always, but sometimes) to let the mando ring a bit more in the background, play along with whatever recordings you can; and as Dix says, play a little EVERY DAY, use a metronome, play every day, and play every day!