Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

It sounds like you are aware that you are scraping videos that are later re-labeled as "unlisted", but you don't mention what you do to mitigate this problem.

Even if it may not be illegal, at the very least it would seem un-ethical to link to private videos like this, and it would seem trivial for you to "re-scrape" your database every now and then to check whether any existing videos have changed from listed -> unlisted, and if they have, remove them.



This logic would require them to re-scrape every video forever, which is unreasonable.

I think a better approach for everyone involved would be to only store references to videos which were posted more than x minutes ago. I'm not sure if they have that information when scraping though.


GP said:

>It seems that a lot of users will upload video which is by default published [and then they change it to private] //

So to avoid that sort of unexpected public-ing (ie publishing) only one extra scrape would be needed. Or, if they knew the period over which the setting was normally changed then they could just delay the scrape until most would have already been changed.

I imagine though, in part, the 'fun' is catching inadvertent publication and morality is no t considered.


It actually has nothing to do with "fun". As I mentioned in my other comment, we don't expose our database publicly and nobody but us can see that a video is unlisted.

It would beat the purpose of our service would we delay our identification, and it would actually require some significant engineering efforts in order to introduce such capabilities into our system with significant economical impact on our business.


We don't expose our database publicly and we have no discovery mechanism.

Also I don't believe unlisted videos are considered to be private. There is a private setting which disallows for public to see such a video.

And finally, it's not very trivial to touch 5.5 billion videos often enough to see if any of those became unlisted.


Has "unlisted" ever been known to mean "private"? I never assumed it was - rather it was just a video that would not appear in searches or recommendations on YouTube.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: