I believe Partridge is more likely to be Red John, mainly for 2 reasons:

1, He is not meant to be the obvious choice.  He has only appeared in very few episodes, but "key" ones-the pilot episode and 2 season finale ones. Episodes which are connected to Red John one way or the other. There must be a reason we only see him only there. I mean where is he in all the other crime scenes?

2. If you take season 2 finale and compare Partridge's voice in the beginning and Red Jonh's at the end of the episode, I believe you will find astounding similarities. Red John's voice is more clear and articulate, but tone and pitch are very alike