Gary started his career at Brighton & Hove Albion, where he made two league appearances during two seasons. Gary had two loan spells whilst at Brighton, joining Eastbourne Borough in September 2004 scoring only 1 goal in just 1 game before returning to the Seagulls, he later joined Conference side Aldershot Town on loan in January 2006, playing just three games.
At the end of the 2005-06 season, he was released by Brighton and signed for newly-promoted Conference side St Albans, the club was relegated at the end of the 2006-07 season, for which he was named as Player of the Year. Whilst playing for St Albans he has played for the England National Game XI. Gary played 16 games for St Albans.
In his time with the St Albans City he made 84 appearances and scored seven goals. Nominated for the Saints Player of the Season 2006/07, he opted to leave the side in December 2007 to join Havant & Waterlooville where he quickly became a regular in the heart of the defence, playing all but two of the remaining league games of the season and picking up two Player of the Month awards in the process.
During the 2007-08, Gary joined Conference South team Havant & Waterlooville, although the player was cup-tied and therefore ineligible to play in the FA Cup third round tie away at Liverpool.
In May 2009, Gary signed to Conference National team Eastbourne Borough.
In an interview with BBC Sport during the 2010/11 season he said the side must keep their concentration and avoid naivety in order to regain their good run of form.
"As soon as you get complacent and switch off for a millisecond you'll be punished in this league.
"We've got to learn a little bit and maybe not be so naive."
He also added: "We need to keep setting the bar higher and higher. I'm sure we'll get back on the training ground and put things right."I really want to close the gate with our goal and defend really solidly. Set pieces are causing us a bit of trouble at the moment."We'll get more and more confident as games go by."
Gary's younger brother, Tommy Elphick, currently plays for Brighton & Hove Albion.