Here's where Americans can go with either a valid picture I.D. or sometimes, a birth certificate:

ANTIGUA AND BARBUDA
ARUBA
BAHAMAS
BERMUDA
CANADA
DOMINICA
DOMINICAN REPUBLIC
GRENADA
JAMAICA
MEXICO
MICRONESIA, FEDERATED STATES OF
MIQUELON ISLAND
NETHERLANDS ANTILLES
PALAU, THE REPUBLIC OF
PANAMA
SAINT KITTS AND NEVIS
SAINT LUCIA
ST. PIERRE
SAINT VINCENT AND THE GRENADINES
VIRGIN ISLANDS, British
WEST INDIES, British

Gathered from the U.S. Department of State. I believe Belize used to be on that list, because I didn't need a passport when I did archaeological work their in 1989.

Ther are also:

Puerto Rico, the U.S. Virgin Islands, Guam, American Samoa, etc. which are part of the U.S.