Stanford was born in 1824 in what was then Watervliet, New York. He was one of eight children of Josiah and Elizabeth Phillips Stanford. New York State Senator Charles Stanford was his brother. His immigrant ancestor, Thomas Stanford, settled in Charlestown, Massachusetts, in the 17th century. Later ancestors settled in the eastern Mohawk Valley of central New York about 1720.Stanford's father was a farmer of some means. Stanford was raised on family farms in Lisha Kill and Roessleville areas of Watervliet. The family home in Roessleville was called Elm Grove. The Elm Grove home was razed in the 1940s. Stanford attended the common schools until 1836 and was tutored at home until 1839. He attended Clinton Liberal Institute, in Clinton, New York, and studied law at Cazenovia Seminary in Cazenovia, New York. In 1845, he entered the law office of Wheaton, Doolittle and Hadley in Albany.After being admitted to the bar in 1848, Stanford migrated with many other settlers, moving to Port Washington, Wisconsin, where he began law practice with Wesley Pierce. His father presented him with a law library said to be the finest north of Milwaukee. In 1850, Stanford was nominated by the Whig Party as Washington County, Wisconsin district attorney.The area where he practiced law, now known as the Port Washington Downtown Historic District, is listed on the National Register of Historic Places.On September 30, 1850, Stanford married Jane Elizabeth Lathrop in Albany. She was the daughter of Dyer Lathrop, a merchant of that city, and Jane Anne Lathrop.The couple did not have any children for years, but their son, Leland Stanford, Jr., was born in 1868 when his father was forty-four. In 1852, having lost his law library and other property to a fire, Stanford followed his five brothers to California during the California Gold Rush. His wife Jane returned temporarily to Albany and her family. He went into business with his brothers and became the keeper of a general store for miners at Michigan Flat in Placer County; later he had a wholesale house. He served as a justice of the peace and helped organize the Sacramento Library Association, which later became the Sacramento Public Library. In 1855, he returned to Albany to join his wife but found the pace of Eastern life too slow after the excitement of developing California.
In 1856, he and Jane moved to San Francisco, where he engaged in mercantile pursuits on a large scale. Stanford was one of the four major businessmen known popularly as "The Big Four" who were the key investors in the Central Pacific Railroad, which they incorporated on June 28, 1861, and of which Stanford was elected president. His other three associates were Charles Crocker, Mark Hopkins, and Collis P. Huntington. They hired Theodore Dehone Judah as the chief engineer.
In 1861, Stanford was nominated again to run for governor of California, and this time he was