It's more than Freedom of speech. It's the Freedom of expression, (peaceful) gathering, and religion as well.

The old rule used to be that one's Freedoms extended as far as the next person's began, and once infringed upon, the person whose Freedoms were being oppressed was right to react. Now, in an assessment of Phelps, we can see that:

He is speaking freely, without NOT allowing others to speak.
He is expressing his beliefs, without NOT allowing others to do the same.
His followers gather in the name of religion.
His demonstrations are "peaceful" in that his followers do not attack others with violence, but rather with hateful speech and slogans.

On realizing this, it should be apparent that he is (unfortunately) not infringing on anyone else's Freedoms, and thus will be immune to public attempts to censure him.

Sad, but true.