If your ssh user has the right priveleges, you can read and write your real users files as the ssh user just fine. I do get that this isn't ideal though.