Enter DiskAid, a free utility for the iPhone and iPod touch that lets you access all of the device drive, not just the music stuff. There are versions for both Mac OS X and Windows and it'll autodetect the device - just plug it in. Once connected, you get a file browser window, which will let you drag and drop files or folders in or out of the device. It'll also let you manage those files and folders - create, rename, delete, etc.
No jailbreaking is required and it currently supports firmware versions 1.1.1 to 1.1.4. Of course there are other similar tools out there like Floola, Senuti and YamiPod but none of those can access the iPhone or iPod touch yet.
Related Posts: iPod -> Folder, Access Your iPod!