[morzh]

I never had to remove the baffle. My grill cloth was not exactly pristine, but not too dirty or ripped.
This said, often times Philco baffles are held to the front panel by several screws; like in 570 grandpa clock.
This holds speaker and the cloth.

See if there are any screws in the corners/along the perimeter of the baffle.

And, yes, to replace the cloth you must remove the speaker. Most everywhere, not just here.