But I have no idea where to start.

I don't want my son to be denied just because I don't know what I'm doing
but, also, I don't want to spend tons of money on stuff that may well be
redundant due to lack of ongoing interest on the part of a small boy.

So, does anyone have any tips as to how I can go about getting him some
fishing time? What do I need? Can stuff be rented?

You know the sort of thing I'm after.

Thanks in advance.