The railway was founded in 1853, consolidating several existing railway companies. In 1968 the NYC merged with its former rival, the Pennsylvania Railroad, to form the Penn Central. Penn Central went bankrupt in 1970 and merged with Conrail in 1976. Conrail was dissolved in 1998 and parts of its system were transferred to CSX and Norfolk Southern Railway, with CSX acquiring most of the old tracks of the New York Central.

There was an extensive network of tracks in the states of New York, Pennsylvania, Ohio, Michigan, Indiana, Illinois, Massachusetts and West Virginia, as well as other tracks in the Canadian provinces of Ontario (southwestern and eastern Ontario) and Quebec (south of Montreal). At the end of 1925, the NYC ran 11,584 miles (18,643 km) of road and 26,395 miles (42,479 km) of track; at the end of 1967, the runs were 9,696 miles (15,604 km) and 18,454 miles (29,699 km).

The Mohawk and Hudson Railroad was the oldest segment of the merged NYC and was the first permanent railroad in New York State and one of the first railroads in the United States. It was chartered in 1826 to connect the Mohawk River in Schenectady with the Hudson River in Albany, providing a way for freight and especially passengers to avoid the long, drawn-out Erie Canal locks between Schenectady and Albany. The Mohawk and Hudson opened on 24 September 1831 and changed its name to the Albany and Schenectady Railroad on 19 April 1847.

The Utica and Schenectady Railroad was chartered on 29 April 1833; because the railroad paralleled the Erie Canal, it was prohibited from carrying freight. Freight service began on 2 August 1836, extending the Albany and Schenectady Railroad's line westward from Schenectady along the north side of the Mohawk River, opposite the Erie Canal, to Utica. On 7 May 1844, the railway was allowed to carry freight with certain restrictions, and on 12 May 1847, the ban was completely lifted, but the company still had to pay the equivalent in canal tolls to the state.