We call it the greengrocers' apostrophe this end. You can buy a pound of potato's or half a pound of tomato's. And how about lending me some of your CD's?

"Albeit" doesn't offend me. It makes me smirk when I see it, because there are far more sensible alternatives (try "though" every time). That makes "albeit" pretentious. Same with "prior to" ("before" works every time) and "on a daily basis" (try "every day" every time). These three examples are all standard English, therefore not wrong, but they are decidedly not elegant English.