Should I fight this person?

Most fights stem from insecurity. If you know you can win then why prove it, what do you owe this guy? Go to a brazilian jiu jitsu gym and train, get it out of your system, you will feel better. Legally you don't have a right to fight, like the saying, talk is cheap. If you fight everyone who says something negative about you that is all you will get done. Your confidense and strength are yours, some guy running his mouth can't take that away.