Infixes are certainly the fewest in number. The Wikipedia article on Infixes mentions 6 or 7, and even that's a generous estimate. (Some linguists would argue that English has no true infixes.)
Prefixes are intermediate in number. The Wikipedia page on English prefixes lists 34 native prefixes, 96 Neo-Classical prefixes, and 3 archaic prefixes, for a grand total of 133.
Suffixes are by far the most numerous. Wiktionary's list of English words categorized by suffix has around 680.
en.wiktionary.org/wiki/Category:English_ words_ by_ suffix
(To get around a bug in the code of this site, I had to add a couple spaces in this URL next to the underscores. Just delete them when you copy-and-paste this into your browser address bar.)
A complete answer to your question would require first setting up good, clear definitions of what will be included and what will be excluded from each of these 3 lists, and then to comb through the literature for more reputable sources than Wikipedia. However, as a first approximation to an answer to your question, hopefully these figures will suffice.