Well, I figured out why I was lagging so hard in the nether and the end. Its forge. Specifically forge has a bug on what they say is "too few video cards to worry about fixing" where it forces my PC to render everything, and won't drop frames. I guess the way minecraft works, if you aren't looking at something you computer doesn't always render it. With this bug everything gets rendered greatly reducing my PC performance (i'm not playing on a potato so don't start).
On top of that it won't drop frames, it forces every frame to load. The result is things that happen server side get backed up to be shown client side for me resulting in a delay. The longer I play and the more stuff that goes on, the longer the delay becomes. So say I'm 5 seconds delayed, and a zombie pigman runs up to me and starts murdering me, my PC will show him walk up to me, let me kill him, then after the delay has caught up I'll eat 5 seconds worth of damage that happened server side. Any internet bumps add to the delay as well, if you guys have an internet bump, you might see a creeper warp several blocks over. I won't, on my screen the creeper will stand there, then move at normal speed thereafter. And I'll be a second or two further behind on my delay.
Forge says I should change my video card settings to prevent this. I say fuck forge, I'm not changing my video card settings for a single mod from a single game that won't fix its bugs.
TL;DR-It was a fun server, but I've reached the pinnacle of what I can do on it, you guys have fun and feel free to loot my stuff. If you find my gravestone in the end, it has 50 emerald ore on it.