Show HN: Downloading a folder from a repo using rust

tsylba · 2025-09-10T11:53:18 1757505198

You guys using convoluted git commands when a single line of subversion works:

svn checkout https://github.com/tensorflow/tensorflow/trunk/tensorflow/ex...

craftkiller · 2025-09-10T17:35:28 1757525728

Yes, subversion wins for this ONE specific use case. It's such a shame subversion is miserable to use in every other use case.

zenmac · 2025-09-10T12:39:20 1757507960

Link is broken

pornel · 2025-09-10T11:11:13 1757502673

There's hardly any Rust in there. It's shelling out to the git command. This could have been a couple lines of bash.

Actually doing this in Rust with lower-level libraries like gix would have been interesting.

zikani_03 · 2025-09-11T13:04:55 1757595895

I've found that using Git libraries directly is usually slow and less ergonomic. I've got another tool (written in Go) here: https://github.com/zikani03/git-monorepo which uses a Git library and is a bit slow.

But I'm willing to take up that challenge and test out gix, I've regained my interest in Rust so the timing is good :)

panki27 · 2025-09-10T10:59:35 1757501975

   $ git clone --no-checkout $URL/repo.git
   $ cd repo/
   $ git sparse-checkout init
   $ git sparse-checkout set subdirectory_i_want
   $ git checkout main

cakoose · 2025-09-10T11:06:07 1757502367

Now I'm curious -- is that here a way to do this that avoids downloading any more than strictly necessary?

The command above downloads the whole repo history. You could do a depth=1 to skip the history, but it still downloads the he latest version of the entire repo tree.

craftkiller · 2025-09-10T12:19:51 1757506791

You could do a blobless or treeless clone https://github.blog/open-source/git/get-up-to-speed-with-par...

Combined with --depth=1 and the --no-checkout / --sparse-checkout flow that the GP already described.

I just tested on the emacs repo, left column is disk usage of just the `.git` folder inside:

  Shallow clones (depth=1):
  124K: Treeless clone depth=1 with no-checkout
  308K: Blobless clone depth=1 with no-checkout
  12M: Treeless clone depth=1 sparse checkout of "doc" folder
  12M: Blobless clone depth=1 sparse checkout of "doc" folder
  53M: Treeless clone depth=1 non-sparse full checkout
  53M: Blobless clone depth=1 non-sparse full checkout
  53M: Regular clone with depth=1

  Non-shallow clones:
  54M: Treeless clone with no-checkout
  124M: Blobless clone with no-checkout
  65M: Treeless clone sparse checkout of "doc" folder
  135M: Blobless clone sparse checkout of "doc" folder
  107M: Treeless clone with non-sparse full checkout
  177M: Blobless clone with non-sparse full checkout
  653M: Full regular git clone with no flags

Great tech talk covering some of the newer lesser-known git features: https://www.youtube.com/watch?v=aolI_Rz0ZqY

vient · 2025-09-10T11:14:16 1757502856

git-archive downloads only strictly necessary files but is not universally supported

https://git-scm.com/docs/git-archive

archargelod · 2025-09-10T11:46:05 1757504765

The rust app just calls a few git commands too[1]

Could've been a shell script[2]

[1] https://github.com/zikani03/git-down/blob/cb2763020edc81e464...

[2] https://textbin.net/ja17q8vga4

vient · 2025-09-10T11:02:29 1757502149

Same can be done using git and tar

    mkdir -p <out_dir> && git archive --remote=<remote> --format=tar.gz <branch> <files...> | tar -xzC <out_dir>

Strangely, github does not support this, so tested with bitbucket.

pharrington · 2025-09-10T13:33:39 1757511219

Since this is a "Show HN," did you mean to post using your https://news.ycombinator.com/user?id=zikani_03 account?

zikani_03 · 2025-09-11T13:03:05 1757595785

I'm not the one who posted, thanks to the OP for sharing though :)

luismedel · 2025-09-10T10:49:33 1757501373

So,

  $ git-down -d bootstrap-dist https://github.com/twbs/bootstrap.git:master dist

is *way* better than

  $ git clone --depth 1 https://github.com/twbs/boostrap.git
  $ cd bootstrap
  $ mv ./dist ~/stuff/boostrap-latest

because: "C'mon, you don't have the time to be doing all that."

And then we wonder how the fuck we end with malware in our systems.

panki27 · 2025-09-10T10:52:21 1757501541

All that's missing is `curl ... | sudo bash` as install instruction in the README

koakuma-chan · 2025-09-10T11:03:08 1757502188

I have a similar problem with hugging face. I do git clone and it doesn't download models. I know it's supposed to use LFS, but I don't know how to make it work, I tried everything. I had to install their disgusting Python CLI to download a model.

0fflineuser · 2025-09-10T12:20:52 1757506852

Don't you just need to install git-lfs https://git-lfs.com/ and then run `git lfs pull` ?

koakuma-chan · 2025-09-10T12:25:52 1757507152

Oh I guess I didn't run `git lfs pull` lol thanks