The keyboard has to have a USB port--remember that there are still some inexpensive synthesizers around that don't have any output at all, or only analog audio outputs. There are also some synth units that are used for live performance, such as the palm sized Kaosillator, that only have analog output.

There are also synthesizer workstations, like the Korg Tritons, which have midi sequencers and even recording software built in, so you can use them to create MIDIs, and you can convert the MIDI to soundfiles, and add analog tracks--

It gets even more complex than this, because there are always new devices that get added to the mix--I've been wanting to buy a new keyboard for more than a year, and I can't quite figure out whether I want it to be a workstation, or whether I want to just do that all on my computer. It's a puzzlement.