A single quote is actually shorthand for the quote special form
'foo
(quote foo)
'(1 2 3)
(quote (1 2 3))
It basically does not evaluate its arguments but returns whatever it is. So instead of the value of foo, it the symbol foo. The list example does not evaluate it either, it would normally try to apply the first element as a function, but instead we get the list of numbers.
While I don't write Scheme, I believe it is Scheme -- one of the goals of the GNU project is to have every GNU tool be configurable with Scheme (which is why GNU has a Scheme implementation as opposed to a Common LISP implementation).
Rust's usage of single quotes comes from CS language theory (lifetimes are denoted using quotes) which I believe comes from the normal usage of quotes in possessive words (James', Amanda's). I don't think LISP's usage comes from the same place (it means something very different semantically).
Lisp's quote (or ') comes from the word quote and the context:
(quote something)
Meaning: take whatever follows and leave it alone. Do not interpret it. If it's a list it stays a list (one of the main uses for this). The single quote ' is shorthand for that. It's meant to be an easier way to create a literal list (and other things) without escaping all the parts of a list that might be misinterpreted as symbols to evaluate (to values or functions).