Yes, by the time IR reach top position, Mohd Rafi was no more.
Besides, the noof songs sung by Mohd Rafi were less in the late 70s. Already he was retiring in hindi films.

Giving songs to Mohd Raji IMHO, is like giving a song to Jesudas

Kishore Kumar - SPB

Why not ? It would have clicked when TMS- IR also could give some nice songs !

Imagine Sendhazampoovil vandadum thendral - by Mohd. Raji !