5mg is TINY and not enough to do you much good, let alone much bad. Given vestibular conditions and anxiety are apparently the best of friends, I'd wager its your general increase in anxiety that's the culprit.
Amitriptyline at low doses is a pretty popular, tolerable medicine. I woudn't write it off until you've been on it at least 2 months at a decent doseage. Waste no time in getting on to at least 10 mg, and I'd suggest 20mg a month later. You may end up wondering why you took so long because 20mg made my life so much better.
There were a few side effects to be wary of at the beginning: dry mouth and vivid dreams. This wore off pretty quickly. At times you may find you bowel a bit 'slower'. That's my most annoying side effect.