Polyphase filtering is less crazy than it initially sounds. Conceptually, you can think of it as: I have this signal in frequency f. I want to resample it to frequency (b/a)*f, where a and b are integers. (You can also do polyphase filtering to resample of non-rational or varying ratios, by essentially approximating towards a rational, but let's ignore that for the moment.) a and b can be pretty large if you want, e.g. a=160,b=147 will downsample from 48 kHz to 44100 Hz.
So what you do to resample a signal (again conceptually), is: 1. Add <a> zeros between every input sample (which repeats the spectrum <a> times), 2. Apply a suitable (long!) FIR lowpass filter so that the signal is bandlimited, 3. Take every <b>-th sample (which doesn't cause any aliasing due to #2).
Now the core of the polyphase filtering idea: We don't need to actually calculate the FIR filter for the samples we don't want in #3. And most of the input values to the filter will be zero due to #1. So instead of storing all the zeros and stuff, we simply pick out every <a>-th tap of the FIR filter and use that on the input signal directly. But since a and b don't line up perfectly, this means we get a different subset of the FIR filter for every output sample; we have a time-varying filter (or a filterbank, if you want). You get <b> different such filters before you're back where you started.
