Greg Clarke, also a vice president of FIFA, was also accused of inappropriate terms in reference to South Asian and gay players.
The chairman of the Football Association in England has resigned over "unacceptable words" he used to refer to black players before a parliamentary committee.
Greg Clarke, also a vice president of FIFA, had apologised immediately for using the phrase "coloured footballers" while discussing racist abuse experienced by players.
But his remark prompted an outcry.
"My unacceptable words in front of Parliament were a disservice to our game and to those who watch, play, referee and administer it. This has crystallised my resolve to move on," Clarke said in a statement.
"I am deeply saddened that I have offended those diverse communities in football that I and others worked so hard to include."
The 63-year-old, who had been in the post since 2016, was put on the spot during the House of Commons select committee hearing by Labour MP Kevin Brennan, who asked him to retract the comment.
"If I said it I deeply apologise for it,” Clarke said. “I am a product of having worked overseas. I worked in the USA for many years where I was required to use the term, ‘people of colour,’ and ... sometimes I trip over my words."
Clarke also came under fire for other comments he made before the committee.
The ex-chairman claimed that South Asian people have “different career interests” than playing the game, and described sexuality as a “life choice” while talking about the lack of openly gay male footballers in England.
"His use of outdated language to describe Black and Asian people as 'coloured' is from decades ago and should remain consigned to the dustbin of history," said Sanjay Bhandari, executive chair of anti-discrimination group Kick It Out. "Being gay is not a 'life choice' as he claimed," Bhandari added.
Former England player John Barnes deplored the fact that the controversy overshadowed the issues of funding for clubs and diversity in football that the committee was discussing. The ex-Liverpool star, who is black, suggested that more tolerance should be shown given that terminology deemed to be acceptable has evolved over the years.
"When I first came to England, I was told that you can't call people 'black', you have to call them 'coloured', so if you're in your sixties that's what you've been used to saying," he told BBC radio.
"If you're a particular age, every now and again you may have a slip of the tongue, but the intention is the most important thing," he went on. "Greg Clarke, he apologised straightaway... so I haven't got an issue with it."
But others said Clarke's comments undermined the FA's position over issues such as racism, representation and diversity.
Former England player Darren Bent, who is also black, tweeted: "Slip of the tongue was it, awful just awful."
The FA said in a statement that Peter McCormick would step into the role as interim FA Chairman until a successor to Clarke was chosen.