Go back and watch the season opener.
In the desert rose I think we see the most convincing taunt by patrick .
Watch the way that jane looks at partridge and listen to sound of his voice. The rest of the season he plays games with the suspects only Partridge do I see what appears deeper and much more emotion behind his interaction . I think jane knows it's him and look at eery feeling Brett has as he's beginning to see the same thing . Think the season opener and season opener to end of red john begin with jane and Brett .