I agree, however some accents are more fun to listen to than others and that's my issue. I don't want to bore people.
Honestly don't worry about it.
I think most English speakers will appreciate the accent and those that don't are weirdos.
One of the best thing about English being the number one worldwide language is the glorious accents.
Also remember that, chances are, you are speaking their language and they can't speak your native tongue at all.
If they are decent people they will appreciate that you have taken the effort to learn their language and will find your accent cute, regardless of how you think it sounds to your ear.