It's not like you have to watch or listen to the video in one sitting. Youtube remembers where you left off on a video, or within 15 seconds of where you left off. Or you could open videos in new tabs and pause them when you're not planning to watch or listen to them.