The veteran film director made the comments in an interview with Spanish newspaper El Mundo. He is in Europe at the moment for the Venice Film Festival, where his new film premiered.
The American filmmaker was asked about the World Cup kiss scandal plaguing Spain, in which Spanish Football Federation President Luis Rubiales kissed Jenni Hermoso without consent after the nation's historic World Cup win.
Allen, 87, said Rubiales should not lose his job because of it, referring to it as "only a kiss".
“He wasn’t raping her,” he told El Mundo.
"It's hard to imagine that a person could lose their job for kissing in public.
“It was only a kiss and it was a friend. What’s wrong with that?”
The comments come a day after the captains of Spain's men's national team publicly condemned Rubiales, calling it “unacceptable behaviour”.
This year's Venice Film Festival has headlined films by Allen, Luc Besson and Roman Polanski, despite calls for the directors not to be included in major filmgoing events due to their histories of sexual misconduct accusations.
Allen's French-financed, "Coup de Chance" is his 15th feature and, first entirely in French, is showing out of competition at the Venice Film Festival.
It premiered on Monday and received a standing ovation however there were noisy protests outside on the red carpet with demonstrations against the screening.
"Coup de Chance" is scheduled to be released in France on 27 September.