HP was founded on January 1, 1939 as a manufacturer of test and measurement instruments, by Bill Hewlett and Dave Packard, with a US$538 investment.
They had both graduated from Stanford University in 1934. The company originated in a garage there while they were still fellows at Stanford. (so their story was more "academic gowns to riches" than rags to riches).
Their first product was a precision audio oscillator, the Model 200A. Their innovation was the use of a small night-light bulb as a temperature dependent resistor in a critical portion of the circuit.
This allowed them to sell the Model 200A for $54.40 when competitors were selling less stable oscillators for over $200. The Model 200 series of generators continued until at least 1972 as the 200AB, still tube-based but improved in design through the years. At 33 years, it was perhaps the longest-selling basic electronic design of all time.
The company name, Hewlett-Packard, was derived from the founders' last names. Had Bill not won a coin toss, the company today might be known as Packard-Hewlett. One of the company's earliest customers was Walt Disney Productions, who bought eight Model 200B oscillators (at $71.50 each) for use in testing the Fantasound stereophonic sound system for the movie Fantasia.
The Wikipedia entry for HP can be found here.
The video below narrates the evolution of Hewlett Packard as a company, from its humble start to growing into one of the most powerful IT companies in the world.