How do you deal with a boss who's crossing boundaries from the get go?

I recently was hired as a temp through an agency for this company I have been working for a month or more. The first week I was there, my supervisor, pretended that he was going to slap me. Through me for a loop! It was strange and unprofessional but I went with it. Since then, I have dealt with him talking to me about his family and other things...it's all strange to me, is this normal or should I be cautious or look for another job? Help! They are also thinking of hiring me perm, but should I take the position if all others are beginning to follow his lead? What do I do?