Kelley was a three-sport athlete at Oxford High School where he excelled in football, basketball and track. He was a first-team linebacker on the Class 4A-6A All-Calhoun County football team after his junior season in 1985 and again as a senior in 1986. He also played tight end on offense for coach Robert Herring as a senior. At the close of the 1986 football season, the Alabama Sports Writers Association named Kelley to its Class 5A All-State first-team defensive team at linebacker. 

Kelley was a Class 4A-6A first-team All-Calhoun County selection in basketball his senior season. In 25 games, he averaged 17.8 points per game and 7.7 rebounds per game for coach Larry Davidson. He scored a career-best 49 points against Saks as a senior. In the championship game of the prestigious Haralson County (Ga.) tournament, Kelley had 34 points and 13 rebounds as the Yellow Jackets defeated Bremen (Ga.). He was named the tournament MVP. 

In track, Kelley was a thrower for coach Tim Britt. As a senior in 1987, he helped Oxford finish first at the sectional track meet by winning the discus. In the Calhoun County meet, he was second in the discus as a junior and again as a senior and was second in the shot put as a junior. 

Kelley became a four-year letterman for Jacksonville State University and coach Bill Burgess. As a senior defensive end in 1992, he was a co-captain of JSU’s NCAA Division II national championship team. In the championship game victory over Pittsburg State (Kansas), Kelley made 14 tackles. He finished with 127 tackles on the season, 60 of them solo stops, and caused five fumbles. The Associated Press named him to its second-team Little All-American team. Football Gazette made him a first-team All-American and its defensive lineman of the year. Kelley finished his career with 315 tackles for Jacksonville State.