Justin was born in San Antonio, Texas and spent most of his life in the Lone Star State with his mother Elaine, who separated from Ernest in 1948. He got his professional start in local clubs during college, and eventually moved to Nashville.

At his father's suggestion, Tubb got a job working as a deejay in Gallatin, Tennessee, where he occasionally performed some of the songs he had written, and made his recording debut in 1953 with "Ooh-La-La." Throughout the 1950s, Tubb recorded steadily, but had only moderate success with his solo efforts.

He did a little better singing novelty duets with Goldie Hill; in 1954, they reached the Top Five with their version of Jim Ed and Maxine Brown's "Looking Back to See," followed with the Top 15 "Sure Fire Kisses."

In 1955, he joined the Opry, and in 1956, he had his first solo success with the Top Ten hit "I Gotta Go Get My Baby."