You will most definitely have to do something, brother.
Running like that without adjusting anything, even if it feels OK, can hurt your bike long term. If you're too lean over long periods of time you're overheating the bike more than necessary and you can also burn your valves. Don't do it.
At least pull the bowl and change the main jet to a 125+/- or something just to be safe. Once you've pulled the carb one time, you'll be a pro at it.
Pilot adjustments, needle shims and everything can come later if you want to really put it off. Just don't ride around like that for too long. You'll end up regretting it and you won't get nearly as much joy out of getting a little bit more power as you could have.
Some useful links:
Dupo's Muffler Swap -
viewtopic.php?f=8&t=19
Needle Shim Mod -
viewtopic.php?f=8&t=15
Jaime's Power Increase -
viewtopic.php?f=8&t=617
In those three threads alone you should find just about everything you need to know about getting your jetting right.
I also worked on some crap a long time ago that might be useful if you like long-winded reading and over-thinking:
Jetting: Part 1 -
viewtopic.php?f=9&t=513&hilit=jetting