Depends what you want to do as other people have said

I have a little Zoom recorder (H2 an older one) which was about £130 when I bought it (probably cheaper now) and is a stalwart for me. Plonk it down in front of you or in the middle of a group of players (or plug it into amp though you would need to mess about with levels or it is too high I found and so bought a lead with a little volume control on) and you will have a decent recording.

Probably need something to clean it up afterwards and Audacity is fine but I never spent much time using it rather preferring Reaper - which can do multi tracking (like other ones) and all sorts of things - as long as you are ok reading the (very good) manual

I used to use (rather sadly) an old akai cassette deck as a preamp and then route it into the computer but bought an M-Audio M-Track for about £58 (for example ) which makes recording things better for me. Plug it into computer stick the jacks or xlr lead in and off you go.

I like reaper because it does offer a lot of ability to clean up stuff and multi track pretty easily when you have got the swing of it.

Like most software it depends what you are prepared to put in time (rather than money) as to what you get out the other end

Three examples that are were easily accessible (not anything wonderful just examples of what you can SIMPLY get out of stuff) -

ZoomH2 recorded outside with the Zoom sitting on a table - hear the scrunch of the pebbles. Normalised in Zoom

Live gig with a keyboard stuck over the top afterwards - in response to the 'minus keyboards obviously' comment :)

Recorded Reaper via m-audio - work in progress. Multi tracked with some midi stuff chucked in. Excuse the dodgy singing and iffy harmonies I have yet to re-record

I used to use Cakewalk Guitar Tracks when doing some stuff for a friend. Same sort of principle of being able to multitrack. My friend was very happy when this was broadcast on local radio so I guess the quality can't be too terrible given the cost was negligible apart from personal time

Cakewalk