Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
Re: Yesterdays post on cli tools being near 100x faster than dedicated systems
3 points by biturd on Jan 20, 2015 | hide | past | favorite | 3 comments
I built sentences from /usr/share/dict with a small one line script I made.

I have as `final_data` as a source file with a bunch of strange sentences in it.

$cat final_data | wc -l 1501920

So, a little more than 1.5v million "records" in the file.

And let's time the operation: $time cat final_data | wc -l 1501920

real 0m0.063s user 0m0.052s sys 0m0.043s

Thats some speedy stuff. Keep in mind, I am running this on a Mac OS X Yosemite MacBookPro (15-inch, Mid 2010), 8 GB (Self upgraded from 4GB) 1067 MHz DDR3, 2.53 GHz Intel Core i5, so by no means at all a speed demon. There probably literally are now faster phones on the market.

I imagine, were I to drop two SSD's in here the second in the CD tray slot and worked the source and destination data from each drive so they are never combating for read and write access, this would be sped up a ton. Or I had a SSD array.

Next up, lets drop off the cat and work right away with the `wc` application. if sure get blazingly faster:

$time wc -l final_data 1501920 final_data

real 0m0.059s user 0m0.044s sys 0m0.015s

faster in every regard. Just for fun, lets pump a grep call in there and see how much it speeds up, slows down, who knows:

* Remember to clear your terminal scroll back when looking at large data like this, or set the threshold to a lower value. Until I did, Mac IS X does not like multi-megabyte text files.

Wow, that was significantly faster than I thought:

$time grep '\.' final_data | wc -l 1501920

real 0m0.705s user 0m0.712s sys 0m0.035s

And, lets do it how some things sometimes happen, but they are just one off's:

$time cat final_data | grep '\.' final_data | wc -l 1501920

real 0m0.766s user 0m0.771s sys 0m0.041s



or crap, basic returns are not part of markdown, and my editor sucks I guess.


Yep. It would probably be easier if you write it in a quick readme.md on github and post the link to that, as it's hard to read with greyed text.


markdown not supported?




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: