Most of New York was sculpted by glaciers.  As those glaciers retreated they left hills in the form of dunlins, deep ponds known as kettles, and many other interesting formations.  Long Island itself is the product of a glacier.  Gorges were formed by the glaciers as they pushed and carved their way south through the state.  Many of them were dammed at the end by sediment and formed lakes (the Finger Lakes were formed this way.)  Some of the dams then eroded away until we are left with spectacular gorges such as Letchworth Gorge straddling Livingston and Wyoming Counties and Watkins Glen in Schuyler County.  Water still runs through these gorges and creates spectacular waterfalls.  There are over 4,000 waterfalls (defined as a drop of ten feet or more) in New York state.  Some of the more impressive examples are Taughannock Falls on Lake Cayuga and, of course, Niagara Falls.