Should you be afraid of ticks?

Is a tick, that disgusting little mite especially when it swells up with your blood, really dangerous? Sure, it's a vector of diseases like Lyme borreliosis and tick-borne encephalitis (TBE), but does that mean we should stress and panic every time we go for a walk?