Did you try to sell your players, currently i tried to sell two of them myself in the last week, and the survey drop from 4-4,5 to 2, or did you sell a lot of player in the first half of the season? Normally the fans need a while to get used to a new player and forget the old one.
I sold my 3 Trainees at the beginning of the season and also 3 of my go0d players because I didn't need them any more. but the second half season I have no transfers or even set for TL because I reached my ideal and just lack of a C that I bought early second half season. anyway my stars are my Trainees who score about 30 points every time they play starter. they are my best players and I did not even put them in TL!
the freaky thing is that it changes between 0.5 and 1 ball constantly and never goes upper!
how about other parameters? such as TV game or Derby? if they get a certain number of balls it remains until next time it happen. for example I won my rival, after that I saw 5 balls at Fan Survey and it still remains and no changes even 0.5. I think if I lost that even by 1 point I would suffer seeing 0.5 ball every time I check Fan Survey :) what that happened about my TV game :)
but about transfers. if you have 5 balls then put your worst player in TL (which one is at end of the bench and never plays and get only 1k salary) suddenly it dropped huge down as you mentioned and it seems never want to rise up again! :)