It sounds like you have no way to get to the Windows CE shell, nor is there any command-line interface. In other words, you're stuck in the device's built-in application on the screen. Is that correct?
(If you could access either of these then you might be able to get past the application it's booting into and install a third-party USB mass storage driver.
)
Other possibilities: does the device have an SD or CF or PC card slot? Or does it have a cable to connect it to a PC? There may be ways to access the file system through these to do the same things I mentioned above.
Maybe if you have a name or model number of this device we can find more information about it online.