It this context of expressing annoyance, it is inconsiderate for the person to be referring to you as always annoying. Even though we all have different level of tolerance to which we can take anything from somebody, it's still unfair on his/her part to do so.
However, for you been called annoying for asking questions in a repeated manner, it is not nice. Owing to the fact that you were only trying to get clarification on something you didn't understand, makes it insensitive on the part of the individual that called you annoying.
In spite the fact that there is limit to human tolerance, so it is to human understanding. It's not everybody that have the same level of understanding. There are those that are slow to uptake while some are very quick in doing so. For this reason, the person shouldn't have called you annoying instead tolerate your level of understanding so that you can grasp what's been explained.