A native of Oxford, Grizzard began playing football at Calhoun County High School, the official name of the high school in Oxford originally, as an eighth-grader. He became a three-year starter for the Yellow Jackets at guard on offense and nose guard on defense. As a junior in 1947, he was third-team All-Calhoun County. In 1948, Grizzard was Oxford’s captain and was a first-team All-County selection. Prior to his graduation in the spring of 1949, Grizzard received the award as the outstanding athlete at Oxford.

After completing high school, Grizzard attended Howard College, now Samford University, on a football scholarship. He then spent two years in military service. Grizzard enrolled at Jacksonville State after receiving his military discharge and lettered in football in 1953 and 1954.

While completing his degree at Jacksonville State, Grizzard helped Gilbert Adams, who had taken over as head football coach at Oxford in Grizzard’s senior season, as a volunteer. After graduation, Adams soon found a way to hire Grizzard as an assistant. When Adams resigned at the close of the 1959-60 school year, Grizzard followed him as Oxford’s head football coach.

As head coach at Oxford, Grizzard had many outstanding athletes and a number of successful teams. His 1960 team claimed the Calhoun County championship before losing to Piedmont in a rematch in the Turkey Bowl. In 1966, Oxford tied favored Wellborn in the county championship game and in 1968 the Yellow Jackets defeated Cobb Avenue of Anniston in the title game.

In Oxford, he will always be revered for the success he enjoyed against Anniston, then Oxford’s bitter rival.

In the second game of the 1962 season, Oxford traveled to Anniston and defeated the Bulldogs 13-7. No Oxford team had beaten Anniston in football since the Yellow Jackets won the first meeting between the two neighbors in 1920. The Birmingham Post-Herald named Grizzard its statewide “Coach of the Week”. Almost immediately, work began to raise the money necessary to build a new home grandstand at Lamar Field. Grizzard was selected by his peers as Calhoun County’s coach of the year.

Grizzard’s Oxford teams continued to enjoy success against Anniston. Over his final nine seasons as head coach at Oxford, the Yellow Jackets were 7-2 against the Bulldogs.

Late in the 1966 season, the expansion of Lamar Field had been completed but the new seats did not arrive in time for Oxford to host Anniston as originally planned. Grizzard continued to lobby for the game to be played in Oxford every other year. In 1968, Anniston came to Oxford for the first time and the Yellow Jackets won 19-0 to open their season.

Grizzard continue to coach through the 1970 season. He resigned his football position but continued to work with students.