Smith spent much of his childhood in Angus and showed great footballing promise playing for Montrose Roselea and Dundee North End. Smith was to return to Edinburgh to pursue a career as a professional. It wasn't to be with his boyhood favourites Heart of Midlothian, but their cross-town rivals Hibernian. The "Gay Gordon" soon established himself as an idol for a whole generation of post-war football fans. Smith had been due to sign for Hearts, but the 16 year old was signed by Hibs and played against their Edinburgh rivals on his debut, scoring a hat-trick in a 5 - 3 victory on 28 April 1941.
Alongside Bobby Johnstone, Lawrie Reilly, Eddie Turnbull, and Willie Ormond, Smith became part of the "Famous Five", the most celebrated forward line in Hibs' history. During the late 1940s and 1950s Smith and the Famous Five ensured Hibs were one of the most successful sides in Scotland, both sporting-wise and at the turnstiles. The League title was captured three times (1948, 1951, 1952), while 1953 saw Rangers pip Hibs only through the goal average system. Smith was less successful in cup competition however, reaching only one Scottish Cup final with Hibs, which they lost 2 - 1 to Aberdeen in 1947.
A recurring ankle injury led to Hibs releasing him in 1959, even though Smith believed that an operation could cure the injury. Smith paid for an operation on the offending ankle himself, and then he signed for Hearts. He was to enjoy immediate success at Tynecastle, winning both the league and the League Cup in his first season with the club.
After an injury-plagued second season with Hearts, Smith was released and again the football writers prepared obituaries for the veteran's career. Smith was to again confound contemporary wisdom, signing for Dundee and forming a part of their league winning side in 1962. At the age of 38, Smith had achieved the unique distinction of being the only player to win the league title with three different teams, none of them with either half of the Old Firm.
In the following season Smith was paired in a forward line with Alan Gilzean. They helped Dundee to the European Cup semi-finals, where they performed gallantly in an aggregate defeat to A.C. Milan. After leaving Dundee, he played briefly for Drumcondra in the Republic of Ireland, before finally retiring at the end of the 1963 - 64 season.
During his playing career, Gordon Smith was capped 19 times by Scotland, scoring four goals. Smith was compared with his English contemporaries Stanley Matthews and Tom Finney, but he did not appear to have a "big-match temperament". Smith had to compete for selection with Rangers winger Willie Waddell, who was capped 17 times in the same period.