In reality, the House "has enough votes to impeach" a president at any time the House majority and the president are of opposing parties. A bill of impeachment only requires approval by a simple majority (50%+1) of House members. If that were all that was involved in an impeachment, ousting presidents for purely partisan reasons would be a commonplace occurrence.

The reason impeachment happens so rarely (only twice in 200+ years, and neither case resulting in conviction) is that the second part of the process, the actual trial, takes place in the Senate, where conviction requires approval by a 2/3 majority of senators. Getting 66 of 100 senators to vote for removal of a president is almost impossible unless there is clear evidence of criminal wrongdoing. The only time in US history there has been enough evidence that a conviction was likely, the miscreant resigned from office instead of facing impeachment.