When someone calls you crazy, they are really calling you insane. Very often, casual situations are utilized with this term.

Naresh wants to invest all his savings in the stock market. The guy is nuts, I tell you.

I wouldn't go out with that guy if I were you. My friends tell me he's nuts.

The term "nut" was first used to describe someone's head in the middle of the 19th century. The reason behind this was that some individuals believed that the form of a nut and a human head were identical. They also discovered a parallel between the human skull and the nut: both were hard, and the most crucial component was located inside, not on the outside. Whatever the reason, people started calling the head a "nut" instead of a head.
Around this period, it was common to refer to someone as being "off their head" or "out of their mind." As time passed, the term "head" was substituted with the word "nut" in the statement, and very quickly the word "nut" came to imply "mad." Everyone who was insane was referred to as "nuts," and a mental health facility for patients started to be referred to as a "nut home".