As commonly understood, "Taking the Fifth" is more a matter of witnesses, rather than defendants, not being compelled to give self-incriminating testimony.

Jones cannot be compelled to say whether or not he saw Smith at an illegal gambling establishment because admitting he (Jones) was there is also an admission that he was doing something illegal. If the government wants to prosecute Jones for his actions, it must create its own case, not have it handed from the witness stand.

As a practical matter, the Fifth Amendment is rarely invoked in most mundane criminal cases because prosecutors commonly grant witnesses immunity from prosecution or favorable plea terms in exchange for testimony. Jones will gladly admit to seeing Smith at that illegal gambling place if he's only looking at 100 hours of community service instead of a year in prison.