Block Island east of Long Island Sound in Block Island Sound (Rhode Island) is quite a rocky bluff. Startlingly so. Here is a photo. And here (gorgeous!) Here is a book with good cover art. USGS 15 minute map.

Must it be on the coast proper? How about upriver a bit, such as the Palisades along the Hudson northwest of New York City?

Oooo, just looking at these images reminds me why I loved taking road trips on my days off when I worked in New York City!

SRS