Not sure how many I had when I was a kid...it was twenty-six years ago. And I wasn't that organized as a kid, so they usually ended up in the same bin, when not put together.
But now I have 46 sets.
Harry Potter: 31 Sets
Star Wars: 10 Sets
Minecraft: 3 Sets
Indianna Jones: 1 Set (The temple at the beginning of the first movie)
And Fort Legoredo makes 46 Sets.