That's what made me not do it. To be fair, I wrote a Github app once where I wanted read access to public and private repos, and I couldn't find anything in the Github API to give me specifically that. I had to request read+write for public+private, which is horribly permissive and gave me access to a bunch of stuff my app didn't need.
Yeah, I wish there were more fine-grained controls for permissions, i.e. just let me access meta-data like stats for repos (because that's all this app really needs).