Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

> an intern who wrote code .... his code had been in production

You get what you pay for, eh?



A cronjob to sync the time instead of a proper NTP client doesn't sound that good either. I know that each place has its own weird things because historical reasons but NTP is quite ancient.


Our sysadmin was being paranoid about how many memory-resident programs we ran. he figured time syncing wasn't important enough to justify a daemon, so he just ran the ntpdate command on a daily schedule. It wasn't that bad - after all, how far can clocks drift in a single day!? ;-)


> he figured time syncing wasn't important enough to justify a daemon

Time synchronization is very arguable THE most important thing on a server (for numerous security related reasons).

How many moons ago was this incident? It's perhaps forgivable ignorance in the 90s... not so much today.


For security? Kerberos usually has a 5 minute tolerance. Are you saying that's wrong? Because if your hardware/firmware isn't literally broken you won't drift anywhere near that in a day.

NTP can't properly fix a clock like that either since it's often capped at adjusting the speed by one part per two thousand. At most, with a consistently wrong clock, that can handle about 30 seconds per day. Any worse than that and you won't see much advantage over ntpdate.


The bigger the clock skew, the harder it is to correlate events from logs. The harder it is to corelate, the more information needs to come from your machine-fueled winnowing stage, to the final inspection by eyeballs.

1-2 seconds is (probably) well within manageable. But, since you now know for SUER that you have clocks running at different speeds, you need to over-estimate the skew. And hope that the daily skewing is approximately constant over time.

So, yes, clock skew can have an impact on your security, because it makes event correlation (and followup on security incidents) harder.


"It makes logs more annoying, and sometimes security needs logs" is a pretty weak connection, though. And it's a far cry from precise timing being "THE most important thing on a server". Is there anything more direct at all?


It's basically a whole bunch of small "make it harder to correlate" issues. If you have a distributed system (which you probably do, if you care about time to the point of considering if daily ntpdate or continually running ntp-or-equivalent is better), you will probably end up with timestamps somewhere in a protocol.

This could be timestamps in a DB server, or similar.

You can then, if you have too-large skew, end up in the weird position that one of "things that have been commited in the DB is not yet showing up on the frontends" (if the time used as a cut-off for the frontend's query is lagging behind the DB server and the timestamp is set by the DB server) or "things that have been commited are not showing up when you query for SELECT timestamp <= NOW()" (if the DB server is lagging, and the timestamp is set by the client).

If that maters, well, that's really a business and data quality issue.

Some distributed systems will also try to figure out what skew you have across the whole system and then end up taking N times that skew, before it can consider data persisted (see for example Spanner ,and probably CockroachDB). If your distributed system relies on timestamps for consistency, and it doesn't self-discover the skew, you're basically not guaranteed whatever consistency guarantees that the distributed system claims to have.

Again, is this important? It really depends. Is it OK of your distributed data store drops some of your data on the floor and lets you clean up the mess? Sometimes, yes, totally. Is it OK if you get uniqueness guarantees violatedl because two thigs got the same unique ID? Again, sometimes, almost-unique is enough.

Fo most people, log correlation is probably the biggest point, though.


Admin like that sound a lot worse than the intern. Unless maybe if it's an intern-admin...


If you are to be paranoid about time, better be paranoid about stepping time. Time discontinuities can trigger bugs in the best of software.


Guess it's time to change the `0` in that cronjob to a `*/12`.


after all, how far can clocks drift in a single day!?

Well, having run a Linux kernel on OpenBSD's vmm: they can drift more than a single day in that time. I did have to resort to using an ntpdate cron job because ntpd just couldn't cope with the time dilation effect. The cron job was configured at * * * * * (i.e. every minute), which roughly translated to once every 180 wall-clock seconds (+/- 30s).


It’s the size of the correction that can cause problems


I've done equally stupid things as a senior dev. The biggest difference, I guess, is that I had to fix the stuff I broke.


Delivering real functionality in production (with code review from your mentor, of course) is absolutely standard at a Big Tech or hot startup internship.


Depends. FAANG interns can make (annualized) income at or near 6 figures.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: