Miller began his athletic career at Mechanicsville Junior High School where he played basketball and softball. As an eighth-grader in 1951, he led the small West Anniston school to a second place finish behind Oxford in the Calhoun County junior high basketball tournament. His size and athletic ability attracted the attention of Bill Farrell, then coaching football and basketball at Ohatchee, and Farrell persuaded Miller to enter Ohatchee as a freshman.

An end in football, Miller was honorable mention All-County as a freshman in 1951. By his sophomore season Miller stood 6-foot-2 and weighed over 200 pounds and new Ohatchee coach Charles Patty often employed him on end around plays. He scored three touchdowns on long runs in a 47-7 win over Ranburne and repeated the feat against Cedar Bluff the next week in a game the Indians won 45-0. Miller was named first team All-County as a sophomore. As a sophomore in 1952, Miller was second team on the small school All-State team named by the Birmingham News.

In 1953, Miller’s junior year, he served as a co-captain. Ohatchee completed the regular season 8-0-1, earning the Calhoun County championship and a berth in the annual Turkey Bowl game against Anniston on Thanksgiving Day. Southeastern Conference schools were already beginning to court him but Miller’s left knee was severely injured during the Anniston game. Despite an attempt to repair the ligaments in his knee surgically, Miller was unable to play again. As a junior, he was second team All-County. Miller was also named first team All-State at end on two All-State teams, the Birmingham News and the Birmingham Post-Herald.

In basketball, Miller routinely scored over 20 points at a time when many games ended with fewer than 40 points scored by an entire team. As a freshman, he was a member of the all-tournament team for the first Calhoun County basketball tournament. He had scored 19 points against Alexandria despite fouling out with more than 10 minutes left in the game. Later that season in the Sixth District tournament, Miller was a member of the all-tournament team. His football injury prevented him from playing basketball after his sophomore season.