A rare high school athlete, Brannen was outstanding in football, basketball, baseball and track. He played three years in each sport at Anniston High School before graduating in 1956. 

Brannen was a two-way end in football and an outstanding punter. Coach Billy Bancroft’s Bulldogs finished the 1955 season 8-1-1. Brannen was named first-team all-county at end in an era when one all-county team of eleven players was chosen. He was also selected to play in the North-South all- star football game. 

In the 1956 Calhoun County basketball tournament, Brannen was coach F.L. (Bulldog) Johnson’s scoring leader in each of the Bulldogs’ three tournament games. He scored 30 points in their 57-56 loss to Oxford in the championship game and was named to the all-tournament team. He was also named to the all-tournament team for the Sixth District tournament. He led the Bulldogs in scoring as a senior, averaging almost 25 points per game. 

In baseball, again playing for Bancroft, Brannen was a strong-armed outfielder and a power hitter. As a senior, he belted a grand slam home run in a 13-0 win over Etowah County champion Emma Sansom in the Sixth District North championship game. When the season concluded, Brannen was chosen to play in the annual Lions Club East-West regional all-star baseball charity game. 

Brannen finished third in the 220-yard dash at the state track meet as a senior. In his junior year, he was fourth in the 440-yard dash at the state meet and was a member of the Bulldogs’ one-mile relay team that set a Sixth District track meet record. 

Brannen accepted a football scholarship from then Alabama head coach J.B. Whitworth, played on the freshman team in 1956 then lettered for Whitworth at end as a sophomore in 1957. After the 1957 team finished 2-7-1, Paul Bryant replaced Whitworth at Alabama. Although many players left, Brannen remained and lettered again in 1958 when the Crimson Tide was 5-4-1.