Run
sp_helpserver SYB_BACKUP
Does it show the status "net password encryption"? If so run
sp_serveroption SYB_BACKUP, "net password encryption", false
and try your DUMP again.
If the option wasn't on, or if turning it off fails to resolve the issue, please post an update.
ASE and the Backup Server communicate via RPCs rather than by a login involving a password, so in theory this error should not happen regardless of the option setting (and it shouldn't matter if the option is on or off). But I have seen a few reports of it happening.