Was I justified?

I was camping these past few days at a State park. Well Saturday morning
comes and I head out to the lake for some early morning fishing.

As I am fishing a particular shoreline and I see this bass boat coming
behind me from the other side of the lake. Two guys pull up and go right in
front of me, maybe 50-75 feet away. I yell over to the guy that I'm fishing
here and that maybe there's somewhere else he wants to try. He responds
with. "Are you fishing a tournament?" "No" I reply.
He states " Well we are and we won't get in your way, we're just gonna work
our way down" again I voice my displeasure at his inabiltiy to grasp the
concept that I'm fishing there. He ignores me now and continues to keep
fishing. I put down the rod and troll in front of him about 20 yards ahead
to a laydown tree in the water. Cast my speedworm to the outer limbs and
feel a solid thump, I set the hook and get into a nice fish, I boat a nice
2.5 lb bass, hold it up and yell this one would be a good tournament fish
huh guys?, then release it and motor away. A few comments I could not make
out from the duo but I didn't care I was happy. The boater seemed a little
more than upset.

Question: Was I right in doing what I did? Or should I have trolled to a
different location considering I wasn't in a Tournament. What would you have
done. One suggestion was to try and see what I could snag from their deck
with my lures. What are the groups thoughts? I would do it again in a
heartbeat.