Sorry I don't know the answer to this one Andy as I haven't used Pocket DVD but have you tried experimenting by dropping the screen resolution back and ensuring you have the frame rate set at 12.5 fps?
At least if you get something working you can find your way up from there. PocketDVD say that you should be able to hit 464 x 272 x 700kbps which sounds like pretty good going.
I'm mainly a Mac user (although I use a XP box for development) and there's nothing as nice as PocketDVD available for Mac that I've found. But for other Mac users reading this I use a combination of ffmpegX or DVDBackup to rip DVD's and then Kinoma producer to transcode them to the Palm. Kinoma is about US $39 the rest is free.
A great bunch of Mac video tools are here :
http://www.pure-mac.com/video.htmlLet us know how you get on