Aha...I'm going to post this here because it took me a bit to figure out...
If in emacs, you start having trouble with vterm not letting tmux properly scroll the window, check and see if you have pixel-scroll-mode or pixel-scroll-precision-mode enabled. They break tmux working with vterm by changing what pageup and pagedown are bound to.