I think this misses the real problem. So many pieces of foundational software like glibc and OpenSSL are understaffed, underfunded, and plagued by terrible code. Go read glibc getaddrinfo: it's a mess!
Rewriting the software in Rust would not solve these problems any more than rewriting it in C++ would. A rewrite would clean up the code, sure, but then you're left in the same situation, only with brand new bugs that nobody has time to fix.
We need to have incentives for maintaining this foundational software.
I think what GP is saying that you'll have problems in logic or other typical programming problems and that it isn't a lack of language features/safety but rather time/money being thrown at these libraries
I imagine the top two reasons unsafe memory access happen is:
1) other complicated logic seeped into the memory sensitive area or causes programmer fatigue
2) memory management is hard
Rust helps with #2 but let's not kid ourselves, what percentage of a library like glibc would be spent in unsafe blocks? Drawing attention to an unsafe area can help though.
As you can see, I'm running around in circles. Just like every discussion about this does. Until an avid Rust user puts their money where their mouth is, we're all just playing Armchair Programmer.
> an avid Rust user puts their money where their mouth is
There are lots of people doing this. Look at crates.io. I decided to write a new fully compliant DNS server and client, trust-dns. I work on it in my spare time (it's time not money), which is hard with two kids.
I've learned a few things. It's hard to get it done. There's a lot to learn from the rfc's. Just because it's Rust, doesn't mean it's easy. There are all the same borrow rules, and I know I have a bunch of TODOs in the code to go fix things like extra clones. The IDEs, while good, are not on par with other languages yet.
Basically it's a lot of work. And I commend the individuals who've paved the way for me with the std library in Rust, but we still have so much more work to do until there is a full OS env like GNU.
It's true. In my DNS code I have zero unsafe code. Though I did need to add some new unsafe code to the Rust OpenSSL library, but that's really because of the FFI to C. That too would be unnecessary if OpenSSL was rewritten in Rust.
Wouldn't that require pulling in the OCaml garbage collector? Much of the reason people use glibc or other low-level libraries rather than a managed runtime is so they don't need to link in a managed runtime.
Does the OpenSSL library allow/plan for an Rust reimplementation (possibly side-by-side for a while) of the C bits?
Seems that would be best practice - assuming the goal is make everything safe Rust.
In theory that's just copying the Rust bridge code and using that as the frontend API for a rewrite. So yes it's possible, you "just" need to rewrite the rust-openssl-sys library.
I think the hesitation there is that everyone has gotten it beaten into your head that writing new crypto libraries is dangerous. It's probably safer to use a library that has tons of eyes on it, but at some point we should do it.
Problem is that if this is not considered beforehand, one might create APIs which are very hard/tedious to replicate without the old/legacy implementation.
But maybe for openssl staying close to the original API is useful, then one could maybe put a C API on top and use it also outside Rust.
Re: amount of low level code needing unsafe blocks, its worth checking out this series that documents creating a bare metal kernel in rust that has lots of type safety and uses shockingly little assembly or unsafe:
C created this wrong perception that you need lots unsafe code, due to how it does strings and vectors.
In all safe alternatives to C, just like Rust sure you need unsafe at the Assembly FFI level and hardware integration like IO ports and interrupt handling.
> what percentage of a library like glibc would be spent in unsafe blocks?
Probably less than you think. What I think the limit is is that there are functions in glibc that present inherently unsafe interfaces. I'm not sure how many but it might be enough that writing libc in Rust is not that helpful.
OP article was specifically talking about Rustbelt as one tool you would be able to use in the future to mitigate the risks posed by the unsafe code you do have.
But in general, I think that entire idea is that as more code moves to memory-safe languages, the less need we have for unsafe sections.
Ideally, once the kernel itself can be written in a memory safe language, everything running on top of that wouldn't need much unsafe blocks. To be honest, this goal is probably decades away, but we should definitely be considering memory-safe languages for systems programming.
If someone wants to pay me to rewrite glibc in Rust, I'd gladly do it, as I'm sure lots of other devs would. It's actually something I've thought about doing as a hobby, but there are a million other hobbies above it on the list.
What we need is a way to coordinate the effort I suspect. Given we have glibc as a standard interface, a site which matched up function tests with implementations and made it obvious where to jump in could build a decent community.
Yeah, I'm not saying that a rewrite is necessarily a bad idea or that it wouldn't improve things (though writing a drop-in replacement for something like glibc in Rust sounds pretty challenging to me). The situation I'm talking about is the one where no one has an incentive to improve the software day-to-day. If we really want to fix the problem, we should also look at how it got that way.
Memory safety problems are the low-hanging fruit of vulnerabilities. A re-write by unskilled programmers will not guarantee the code is safe, because they will introduce many other kinds of vulnerabilities.
Nobody is claiming that Rust prevents all security vulnerabilities. The claim is that memory safety problems are severe enough that a rewrite is justified.
(Incidentally, there are other classes of vulnerabilities that Rust and/or its libraries heavily mitigate, such as deserialization issues along the lines of the Rails YAML bug.)
By removing memory safety bugs, it allows reviewers to focus on the more important sources of vulnerabilities - the things that the compiler can't check. On that front, it is also worth noting that Rust has a much richer type system than C or C++, and can be leveraged to craft domain models that catch many more bugs at compile time.
With all of that said however, I agree with you that it would be foolish to think that a rewrite would not be open to lots of potential vulnerabilities. At the moment might be prudent to continue to use and support the current libraries, but to also support efforts like rust-crypto in order to prepare for the future.
Rust's improvements over C don't just consist of memory safety. It also provides an excellent type system and the ability to statically check for concurrency bugs at a higher level than just memory safety, for instance.
None of these things (except maybe some of the concurrency stuff) are quite new to Rust. It's just that C is such an old language that stuff that other languages have been doing well for decades were not available to systems software.
> Part of what makes Rust awesome is it forces correct...architecture.
If you believe that, and follow it, then you will end up with a lousy architecture. A correct architecture cannot be designed without knowledge of the problem domain.
That's the problem with security too: if you believe the language will keep you safe, and stop thinking about security, your software will be less secure than before. There's plenty of insecure Java code out there.
> That's the problem with security too: if you believe the language will keep you safe, and stop thinking about security, your software will be less secure than before.
And if you use "it won't be perfectly secure under X mitigation strategy" as an excuse to avoid doing X, your software will also be less secure.
Agreed, the hope is that the combination of Rust + a rewrite, you'd be in a MUCH better place than where we are now.
I think an interesting question is, why are there so many contributors to the Linux Kernel in comparison to something like glibc? Both, I'd argue, are equally foundational.
Speaking only for myself here, but glibc is almost completely unintelligible. I'm not sure if the Linux kernel is as bad, but with glibc anytime I've gone looking for even a trivial function I've regretted it.
A huge chunk of the work going on in the Linux kernel is supporting specific hardware (device drivers, that kind of thing), which doesn't affect glibc and other core userspace things so much. That said, even excluding that, the kernel probably does see more interest and activity than glibc.
You mean other than the fact that it's still running on top of a kernel, BIOS, and other firmware that was ALL written in C or Assembly? Yeah, totally safe /s.
This is why I personally think the OSS community should be more encouraging of "salary" style funding. There is a loud, poisonous minority of advocates that seem to think that asking for money is just plain evil. That attitude is a material disservice to OSS in general: it makes OSS weaker and less capable, for the sake of some weird puritan reflex.
N.B.: I have never and will never ask for, nor accept any income from OSS work: too much shit-flinging and absurd expectations. But as a _user_ I want OSS, for its own sake, to re-evaluate its attitude towards money.
There seems to still be too many people who cling to the romantic notion that Free Software must be produced for free, as a pure labour of love.
It's absolute nonsense - so many of the most successful free/open source software projects are those with salaried employees working on it, and this has been the case for many years. The problem today is of course finding a way to fund open source contributors whose projects aren't quite aligned with the needs of a major company.
What's wrong with continuing to pay? All the time companies pay for support for open source software because they want the "insurance" of having someone to call when the $hit hits the fan.
You don't want to pay anymore, go back to glibc...
True of an individual company or business unit and its major tech decisions, but businesses ultimately die. Thus we have a world where Cobol coexists with Node.js - companies saddled with old software never pay for a new system until it's "beyond too late". But a company with a greenfield project can come in and use whatever hot, hyped-up thing is out there, and it usually in their direct interest if it's free, because at that point they probably aren't terribly concerned about software quality.
Firstly, I want to say I agree with you that these types of projects require more respect and financial incentives. That said, I think its worth noting: Rust fundamentally democratizes systems programming.
Rewriting in Rust, increases the contributor pool for these project from some x to some y. Where x is the subset of people in [1, 2, 3, 4, 5] and y is the subset of people in [1, 4]:
1. Who are eager and have time to help
2. Who know manual memory management really well (also diligent enough to use that knowledge appropriately) [Made redundant by compiler]
3. Who can successfully build, link and test C code on multiple platforms [Made trivial by Cargo]
4. Have appropriate domain knowledge
5. Are not afraid to refactor "working code". [Made better through first class testing, the compiler and community mindset]
There is also the aspect that experts get a dramatic boost to their productivity, reducing their time needed, and therefore reducing the level of incentives needed.
The compiler doesn't obviate the need to understand memory management. Anyone who doesn't understand ownership and borrowing will have really hard time getting rustc to accept their code in the first place.
What Rust's type checker does is make up for the human inability to be perfectly careful. Even the most diligent programmer will make mistakes or forget something. (This is a fact of life. It can't be fixed with present-day technology.) Relative to C and C++, Rust enlarges the class of “stupid mistakes” that can be detected automatically, and this is always a good thing, no matter how knowledgeable and careful programmers are.
We've actually had feedback from some startups using Rust for low level stuff (where their teams comprise mostly of Python/Ruby/JS devs) that Rust teaches systems programming and memory management quite well (since you're forced to work with it), unlike C++.
> Rust teaches systems programming and memory management quite well
Couldn't agree more. Incidentally, this is also what I dislike so much about C: it does a very poor job of telling the programmer what is wrong with their program. Learning systems programming with C feels (to me, at least) like learning to walk by crawling in the dark.
>We need to have incentives for maintaining this foundational software.
It's my opinion that contributing to free software is a public good, and so we should have state-funded employees whose full-time job is to work on this software. I think something like this is already done in places like France and Europe, but to my knowledge the majority of paid-for American contributions to FLOSS comes from corporations.
There is a lot of public funding of FLOSS software through universities. Unfortunately, the funding model doesn't work well for big projects or projects lasting more than a few years. A lot of great stuff is developed using public funds and then abandoned when the developer graduates.
* Shudders * In recent years the shifts reduction of public funding levels combined with a cultural shift toward emulating business models and proving ROI has, IMHO, stilted creativity, exploration, and the creation of basic fundamental science. An analogous situation would be tons of grant money spent on Rails-like frameworks (because it's 'hot') while fundamental projects like "rewriting a safe glibc" would be deemed lacking in utility.
Maybe do both? Rewriting in Rust and increasing incentives.
Writing security critical software in c++/c is not the sensible thing to do in my opinion (even though it is the standard right now). Humans in general cannot handle it without making a lot critical errors. And the people working on those projects are already really smart in general. This has been shown again and again.
If c++ was a car, it would force you to place your foot in a special position in between breaking and accelerating. If you do it wrong, your foot will get squashed in the exposed drive train.
Of course for the point of the argument I am exaggerating, but I really applaud the efforts Rust has made in order to advance the security of software.
There isn't one "real" problem. You are talking about one problem, and rust solves another. I don't see any reason your comment needs to be framed as a contradiction to the OP.
Maybe 20 years from now it would get into the same situation, but the point is in those 20 years it could be a lot easier to maintain. Also, the whole point of Rust is to eliminate some classes of bugs, which means there would literally be orders of magnitude fewer bugs to deal with.
So I don't think that saying there will be bugs in the new libraries, too, is fair criticism.
Also, what's clear is that Linus Torvalds' "just don't be stupid and write incorrect code" policy doesn't work. We need to think of security from a design point of view, not rely on your average programmer to write academically perfect code as a security policy.
It's pretty much unheard of, but nearly every open source project would benefit far more from a week of watching potential contributors trying to get up to speed than it would from making sure the project roadmap is delivered a week sooner.
That's my claim for projects in general, and what you're asking for re Rust is a little different, but the approach can applied there, too. Focus on everything from navigating the project website and other docs, to interactions with the tools that make up the "ecosystem", to programmer expectations that could shape the language (with the latter actually being the least important).
Don't trust users to self-report. Don't overestimate your ability to make a combined diagnosis and prescription, and don't underestimate how much silent resignation is killing your adoption rate. Observe and work off real notes.
This is the most insightful comment in the discussion. It should have been obvious, but it wasn't until you posted it.
First, design a sound language that solves problems other languages don't (done). Then, optimize for adoption ... because the world only benefits from that new language once adoption really takes off.
Rust has done half the job really well. And a decent start on the second half -- mainly because the first half was done so well.
But if the Rust team were to prioritze adoption the way the parent post espouses, they'd be changing their priorities significantly, and it's probably the most effective way to make sure Rust fulfills its potential.
We have done a lot of things to prioritize adoption, though we could do better.
We have done some almost-studies like this: we had a series of conference calls with companies using production Rust specifically to work on their pain points, for example.
Rust is nice. But the article miss one important point. Modern software engineering is a collaborative effort (well, it has always been, afaik). I write code for others to understand. I find Rust code significantly hard to read (I am saying this as an experienced C/C++ programmer and as an intermediate haskeller). Yes, one can write write-only code in any language. I think it is extremely important to write code that others can understand and imho, it trumps all other reasons. Not trying to underplay the value of type safety. It is fantastic. My Haskell programs have much much less bugs and I tend to get it to work, much faster than when I do C/C++ code. I would expect some of that goodness to be in Rust programming as well.
It would have been fantastic, if Rust took only a few steps further from Cyclone. I also wish that Rust authors stop adding more and more features. The target audience, namely low level system programmers, tend to work close to hardware and think along those lines, so bombarding them with type theory concepts is only going to shoo them away.
And as a Rust and Python programmer, I find C++ hard to read (I can never shake the feeling that the ampersands are always in the wrong place), and I'm nearly hopeless at deciphering Haskell. :P I suppose this is just something that comes with experience.
Haskell syntax is really nowhere near as complex as C++'s or even Rust's. It's not indecipherable, it's just different and not C-like. It's in the same family as SML, OCaml and F# so if you learn one of those, the others come easily.
I suspect Haskell's syntax isn't as much of an issue as the typical style that Haskell code tends to be written in; really short, abbreviated variable names, and very high code density.
> I also wish that Rust authors stop adding more and more
> features.
I'm not sure what this is referring to. Rust hasn't added any features since the stable 1.0 release last May, and I also don't believe it added any features in the six-month beta period prior to that. And it's not an especially feature-heavy language in the first place, likely comparable in size to Python (e.g. a medium-sized language).
We have and will add language features; though we haven't added any HUGE things in a while. You are right that RFCs contain language changes. A lot of them are small. Some of them are big, and there's certainly more big ones on the horizon.
The most recent one off the top of my head is the ? syntax RFC, which should reduce the amount of error-handling boilerplate.
The big ones though are:
* Specialization. This will allow you to write ultra-performent generic code by special-casing when you have the knowledge. It also will be a building block for the next few features.
* The bag of features colloquially known as "inheritance", though that's not really a good name.
* Non-lexical borrowing, and various other borrowck usability improvements.
* Abstract return types, which will allow for you to remove some allocation and make certain type signatures much better.
* Incremental compilation.
Those are the ones on the short list. There are others too, like higher kinded types.
Great. They all look great. The ones like higher kinded types are the ones that I am highly suspicious of. The typical C programmer who pokes around hardware data sheets and writes drivers and stuff will likely not pay much attention to Rust if such complex features are added.
Also features like that makes me wonder if you folks are really targeting C/System programmers. Sorry if I sound negative, that isn't my intention.
Systems programming does not preclude a language from having a good type system or higher level abstractions.
A good example is Phil's OS. http://os.phil-opp.com/. He's designing an operating system in Rust, and it has surprisingly few "unsafe" parts. The rest of it is made safe using some neat zero-cost abstractions.
Higher kinded types isn't going to happen anytime soon. It's something that crops up often in discussions when people want to model something complicated with the type system; but it's a very nontrivial feature that would probably need to wait for Rust 2.0, if ever. Even if it would exist, you can just not use it. Like most of the other features being added.
> Systems programming does not preclude a language from having a good type system or higher level abstractions.
I didn't say that system programming does not need a good type system. I am also not saying that it is impossible to design operating system software with Rust.
As someone who had been on both the sides of the abstraction and having closely interacted with the system programmers, I think it is just too hard to convince them to use Rust. It is not just about the language alone, it is about what minimum you need to bootstrap a system (among many other things).
Anyway.. good luck to the OP in "rewriting everything in rust" and also getting it to the same quality/feature parity as others in the game and also get others to use it as well.
Yeah, I'm just saying that "higher kinded types" (which, again, Rust isn't getting anytime soon), does not make Rust something that isn't targeting systems programming; in response to "you folks are really targeting C/System programmers".
Many of Rust's designers are experienced systems programmers. A _lot_ of effort goes into making it systems-ready.
> it is about what minimum you need to bootstrap a system
Rust works on any target LLVM compiles to, and you can opt out of the standard library if you're writing baremetal things. There already are people writing low level Rust things.
But yeah, I know the skepticism you refer to; seen it in action before. But in the case of Rust I've rarely seen any concrete points being brought out (aside from perhaps "LLVM doesn't target enough things", which is fair). The language developers do try to take input from everyone and make it more systems ready; but there really isn't much that can be done when there isn't any input other than a strong preference for C.
They are specifically there so that you can write zero-cost abstractions. Yes, people who are hardcore C programmers may not have experience with more advanced type system features, but it's all in service to the goal of writing expressive code that goes as fast and as safely as possible.
As a practical example of higher kinded types, you can't say "this function takes an Rc<T> or an Arc<T>. I just want something refcounted, but I don't care about how." HKT would get you there.
Another common problem it would solve is "I don't want to write both a &T and &mut T version of my function."
Another gripe I have is that the Rust Book examples are all targeted at non-system programmers. It would have been nice if certain programs written in K&R book or from one of those books from the Bell Labs Unix folks (like "The practice of programming" or "software tools") is written in Rust as a demonstration.
You should be much happier with the second edition of the book.
(One of the reasons it's this way is largely historic: so much was changing in the lead-up to 1.0 that I couldn't add many examples, as things kept changing out from under me.)
Step 0 - write a code generator that just generates rust code that call the C versions of glibc symbols ( based off of the publicly exposed API in the library files). Get test suite to run
Step 1 - Make that rust code generate a 100% drop-in replacement fro the C symbols (maybe this requires injecting version numbers)
Step 2 - write the `getaddrinfo` replacement in Rust
???
Step 4 - have all of glibc ported
EDIT: Does anyone know of any gotchas with linking that would cause this to happen? Seriously considering taking a day off to do step O.
For step 1, you'd need to carefully look at glibc's ABI, which is even more complicated than its API due to backward compatibility and symbol versioning.
Yeah ABI compatibility might be problematic, my currently plan is to LD_PRELOAD what I have to test it for speed. I have the feeling that rust could mock the ABI and serve as a general replacement, but first things first is it even fast enough to use at the lowest levels of the system.
The mere existence of getaddrinfo is a symptom of the disease. Just look at it. It lacks a clear interface. It's designed for calling code which is like, "Gee, I don't know exactly what I've got. Maybe it's a local IPv6 address, maybe it's a remote hostname. But I might want to bind it. Or maybe connect to it. I'll get back a list of these things which might be useful. Most likely I'll choose the first one or maybe write an ad-hoc filter to pick the one I actually want." Systems programming doesn't have to be this crappy.
I get it, the idea is to replace this core low-level library with something you could maybe run C programs on top of. But damn, that is a crappy interface (which is specified by both an ISO standard and an IETF RFC so I must be an idiot, right?).
I'll be thrilled when Herb finally releases his lifetime enforcement tool, but until then it's too soon to celebrate. The mystery surrounding the details of Herb's approach still raises unanswered questions regarding the soundness and practicality of such a tool.
Furthermore, the tool in question does nothing to improve C++'s capability for safe multithreaded programming, which IMO is Rust's actual (yet so often overlooked) ace in the hole.
I don't believe it is, unless something's changed. To quote MSDN:
"The package currently contains checkers for the Bounds and Type profiles. Tooling for the Lifetime profile demonstrated in Herb Sutter’s plenary talk (video at https://www.youtube.com/watch?v=hEx5DNLWGgA) will be made available in a future release of the code analysis tools."
I don't have access to a Windows machine right this moment to check, but a cursory search of the internet gives no indication that this tool has been released in any subsequent update. If it had, you'd think there'd be some fanfare, or acknowledgement, or documentation, or experience reports from users, or anything.
Extra-lingual static analysis tools are OK at finding bugs. They're not good at proving the absence of whole classes of bugs; that is what we should be striving for, and what Rust and other languages can provide.
So what I'm looking for from Herb Sutter is not just a set of good guidelines and tools to check for them, but also a proof --- or at least an argument --- that if the tool finds no errors in a piece of code then that code cannot be the source of memory safety bugs.
I agree, the biggest issue with extra tools is that they require additional effort to use them.
After all, lint was created to compensate for C's unsafety in 1979 and up until clang's introduction of static analysis, barely unused in the industry.
Still C++ isn't going anywhere and is the only native language with first class support in all mobile SDKs, so anything that helps improve its use is welcome.
I won't dispute that the vast majority of C++ developers seem to be indifferent to static analysis (look how many decades it took John Carmack to come around...), but the initial video presentation did cause a noticeable stir, and I would expect the actual release of the tool to incite a comparable reaction.
Nowadays I only use C++ for hobby coding between Android and WP nowadays, or when I need to step out of JVM/.NET worlds, so I missed to follow up on it.
For most applications, moving to modern C++ is going to be a better option than rewriting in Rust.
I wouldn't bet on that.
Companies which go with a language like C++ don't do it for "safety" or "performance" or reasons like that (at least, most of the time they don't). They go with it because they can get something written once, then run it for the next half-century and only have to spend engineering time on new features and the occasional critical bug. Telling them they need an intrusive rewrite of their entire codebase to use modern practices will simply result in either (A) no action taken (most likely) or (B) the rewrite happening in a language that isn't C++.
This is incidentally what happened with quite a few significant Python codebases in the 2/3 transition -- rather than rewrite their code to Python 3 standards, they said "eh, we're rewriting anyway, let's go to Lua" (most often, though some other languages were the "let's go to" option as well).
I don't know that people don't choose C++ for performance reasons or for easy manipulation of low level stuff.
Games, high performance computing, embedded and legacy software are the main applications of C++, aren't they?
Some people on this website seem to blow the python 2/3 transition fairly out of proporotions.. Python 3 is really extremely similar to Python 2, and there are great tools to convert Python 2 code to Python 3 and vice versa.
Is there a great tool to convert Python 3 to Python 2?
I've tried Pasteurize (part of python-future) but it just leaves the "yield from" statements in place, giving syntax errors. And 3to2 does even less, and seems to mostly be a framework for building tools like Pasteurize.
I know this isn't easy, but JavaScript people do it with ES6 transpilers, and ES6 is more different from ES5 than Python 3 is from Python 2.
The two libraries I've created both use a combination of 3to2 and pasteurize to convert the python 3 code to python 2.
The only issues I've run into are new error types that I could not use.
But there seem to be quite some issues open: https://bitbucket.org/amentajo/lib3to2/issues?status=new&sta...
In case you want to see the scripts I use to change the python3 code to python 2 and make it work with pip:
Judging from comments I've seen here in discussions about Python 3, mostly companies who bet on a "write once, run forever" strategy and thought Python would never change.
C++ still can't prevent things like iterator invalidation which are impossible in Rust. While I like modern C++, Rust really addresses a lot of problems with the proper design from the ground up, which C++ can't do. What Rust so far lacks is better OOP mechanisms. C++ beats Rust in that. For instance Rust is still missing something like virtual structs.
Note that while many OOP patterns are hard to express in Rust, there are equivalent patterns that handle the use cases.
To give some context on those blog posts; they exist to figure out how to add single inheritance to Rust. One of the strongest arguments for single inheritance in Rust is "Servo and similar things need it to model the DOM". I.e; the strongest argument out there is that modelling cross-language things with languages that _do_ have single inheritance is hard.
That's not a very strong argument. That's a pretty niche use case (which Servo itself has moved past; we have a setup emulating inheritance that works pretty ok now).
So while I do agree that Rust can't model these patterns easily; I'm skeptical that that can be an issue for the large majority of Rust users.
> while many OOP patterns are hard to express in Rust, there are equivalent patterns that handle the use cases.
May be there are, but they far from trivial, and with the current level of documentation they are in the realm of some obscure hacks rather than straightforward part of the language. Which is a disadvantage. May be when The Book will cover OOP topics in depth and explain such patterns, or there will be some other dedicated documentation on this topic, things could become more clear.
> current level of documentation they are in the realm of some obscure hacks
I don't think so. Rust encourages a certain style of designing code, using enums and traits. Almost all the time this suffices.
My comment was at the macro level; I'm not saying that an OOP pattern will have an equivalent in Rust, I'm saying that if you look at the larger use case you can design your code in idiomatic Rust to avoid it.
> I'm saying that if you look at the larger use case you can design your code in idiomatic Rust to avoid it.
As I said, may be you can, but it's far from trivial and unless you know how to do it already, coming up with such approaches isn't something that one who just learns the language would be focusing on. You yourself said above, that it's hard. And unlike languages like C++ where OOP approaches are very well documented, Rust documentation lacks such kind of information at present.
I said modeling single inheritance is hard. That's a very niche use case; basically only crops up when writing an interpreter for a language with single inheritance (and in cases like Servo's DOM)
> unless you know how to do it already, coming up with such approaches isn't something that one who just learns the language would be focusing on
Again, it's not a thing that you do as a drop-in for inheritance. I'm saying that the regular approach (i.e. using traits and enums idiomatically) to designing your application in Rust will cover the use cases for inheritance, differently.
Basically, don't try to write Rust code as if it's C++ or Java and stuff should work out.
I don't mean one has to copy ideas from C++ or Java to Rust verbatim, but having some explanation on how to use Rust OOP abstractions for solving common programming problems would be useful. I don't think current Rust documentation covers that well.
The book will never cover OOP directly, or at least, the book will always attempt to show Rust as Rust itself, not as in comparison to other languages. "OOP Patterns in Rust" is a very different book.
That said, the second draft of the book is placing a lot more emphasis on understanding how to write Rustic abstractions.
> "OOP Patterns in Rust" is a very different book.
Then may be there should be one like that :) May be it's just my personal impression, but I feel that such kind of documentation now is lacking.
OOP in Rust shouldn't necessarily focus on comparison with other languages (though explaining key differences would be very helpful). But it at least should explain common use cases that OOP expects the language to cover and approaches that Rust proposes for them.
I believe they're equally expressive, but the existing trait approach has a different run-time/memory profile. The issue linked in the parent comment has some info/links, and, http://smallcultfollowing.com/babysteps/blog/2015/10/08/virt... is the latest word on the subject (including various links).
Sharing data layout between implementations of a trait is much more manual and tedious in Rust than it is in something with inheritance. You can often change your design to avoid that issue, but Servo, for example, needs to implement the DOM, and that's just the best way to do it.
But the core guidelines are optional -- which means that you'll still have the entirety of C/C++ footgun hell to watch out for, and decades of outdated teaching and learning materials. And very large codebases which could not be easily ported to a compiler that enforced those guideines.
I will be happy if Rust's main impact on history will have been to showcase how to get those safety features into a practical language, and by extension improvements to C++, but I still think that the steps C++ is taking seem a lot less safe than what Rust has achieved by completely ignoring backward compatibility with C++98. See, for example, the compiler-enforced concurrency safety that Rust provides. I haven't read the core guidelines recently or in depth, but IIRC that's not a stated goal of the project.
Looking at the presentation it looks like you can borrow a non-const reference (in rust parlance) several times, so there's no concurrency guarantee.
You would have to break compatibility with too much existing code to get the same level of strictness as Rust, so that's reasonable, but yeah, it's hard to imagine that C++ will be as safe as Rust. I'm glad these features are being pushed through though.
Absolutely, I didn't mean to say that improving C++ is bad (we sure as hell need it, esp. given how much of the world runs on C++), and C++14 and 17 have many exciting things coming. I just didn't think it was fair to say that the new standard(s) "steals nearly all of Rust's safety thunder," when several core promises of Rust's safety model aren't possible in C++ if you maintain backwards compatibility.
Doesn't even have to do with concurrency. This leads to iterator invalidation and many other problems in single threaded code. In fact, Rust's "standard" concurrency guarantee (with regular, non-scoped threads) doesn't deal with the mutable aliasing rule; it works with the ownership rule more or less exclusively. Even if mutable aliasing was allowed in Rust, you could still build the same safe threading system assuming that borrows are still scoped[1].
The mutable-alias guarantee is more of a "don't let the rug be pulled out from under me". While this is something that's more obviously a problem in threaded situations where a mutation from a different thread can pull the rug out from under you, a function call in complex code that mutably aliases can cause the same issue. The classic example of this is iterator invalidation; but it works with any type which contains a variable amount/type of things, and severe logical (non-memory-safety) issues can be caused with anything with invariants.
[1]: When I say "safe", I'm assuming that in this situation (the lack of rules against) mutable aliasing itself isn't causing any unsafety elsewhere that transitively leaks down and causes thread safety to go kablooey. As such it's a necessary and mostly sufficient system of rules; removing one rule will probably make the whole house tumble. However, the point was that Rust's thread safety doesn't directly derive from the mutable aliasing rule.
It remains to be seen whether that approach will guarantee memory safety. Last I checked, it had gaping soundness holes related to aliasing. (Note: most of the complexity and innovation in Rust is about controlling aliasing.)
When all the soundness holes are closed, then we'll have to evaluate whether it's usable to program in. At the very least, Rust is years ahead here.
Even if those hold up, you have to consider whether rewriting your C++ code to fit the safe subset is actually worthwhile, compared to rewriting in Rust which gives you additional benefits.
It already has a turing complete pure functional sublanguage in it's templates. Probably closer to the worlds worst ML than a lisp, but it's still along the same lines.
This is one of the psychological barriers that keeps me from even trying to learn C++ - every time I hear friends talking about new C++ developments I can't help but feel that modern C++ is so incomprehensibly large that I'm just left paralysed.
Modern C++ is not that huge, it just has lots and lots of weird corner cases and gotchas due to backwards compatibility. You can get reasonably proficient in a couple of weeks, just like any other programming language. You'll just be left with the nagging feeling that you don't understand _everything_ that might happen in your code.
Is it exception safe? Are my iterators always valid? Is this undefined behaviour?
So better keep a copy of the C++ standard open and keep reading whenever you feel unsure.
Full support for Concepts[1] would also further add to the compiler checks as well. Although I believe this hasn't be confirmed when it'll come into the standard. Essentially, they're just Rust Traits.
You misunderstand the role of Rust's standard library. For "libc type roles" you'd be using libcore, which is a subset of the standard library that does no allocation whatsoever, and allows you to define your own allocators and do whatever you like on allocation failure.
Based on my reading, only the std library has the behavior unsuitable for a libc type role. I would think a libc replacement would use #[no std] and have the finer grained control.
> Obviously on OOM you usually want to abort _something_. It's just not always the entire thread or process.
Surely you want to abort _some dynamic extent_. The fact that it's (currently) a full C thread is an implementation detail that can hopefully be improved. Requiring manual OOM handling on every single operation is not a scalable solution and will lead to bugs due to laziness or fatigue.
Exceptions aren't necessarily bad/evil. A lot has been learned about the challenges of exceptions/conditions/effects. Language designs will evolve to have better versions of what is a foundational idea in the theory of computation.
In the meantime, a try/catch block is only a small macro over thread::spawn, thread::join, thread::catch_panic away. It would be nice if the language supported that without the weight of a full thread.
SIGABRT doesn't necessarily mean OOM, and in any case if you're prepared to monitor child processes for aborts then you'll be perfectly fine with the fact that Rust's default stdlib aborts on OOM, which is the context of this thread.
It's certainly possible to incrementally rewrite an OS distribution in Rust, or even smaller projects like glibc.
Anyone want to set up the infrastructure so it's easy to take a library or a .c file and start hacking, and see if you still get a working OS in the end? I suppose part of the problem is defining "working OS" -- I'm not sure I know of any great test suites that check to see that a GNU/Linux distribution continues to in GNUy/Linuxish ways, which is usually what you want for a large-scale rewrite project.
It is GNU LibC, even if your rewrite is GPLv1337, the toolchain isn't "free enough" for them. You will be "stopped by verbal force" in the sake of preserving an inferior, but "free" core library.
How I wish people understood this when we were comparing Ada, Modula-2, Modula-2+, Modula-3, Pascal dialects, Oberon, Oberon-2, Active Oberon, Component Pascal,... to C.
Apparently we were the fools.
But this will never happen in UNIX OSes.
The culture is married to C, regardless how many memory corruption exploits per days might get a CVE entry.
The syntax is killing these languages. Sure it's superficial, but programmers are human beings and I doubt most people find it appealing to voluntary stare at and type PROCEDURE BEGIN END and other LONGCAPITALIZEDWORDS...
I don't know much about glibc, but since you can call rust from C and vice versa, it seems like you could reimplement a few functions at a time, incrementally phasing out C code and replacing it with rust. Assuming, of course, that you agree with the author's thesis. This approach lets you reuse the same tests and do things like benchmarking etc to make sure the implementation is performant and compliant.
This is exactly how Firefox is approaching it. Identify the most security-critical bits, wall them off into their own component, and rewrite it in Rust.
Yes! And the reverse has happened with Servo. Originally it was mainly a bag of C libraries with some layout code, then gradually they began swapping the pieces out for the Rust counterparts. This is how new software can be written - bootstrapping off the C infrastructure, then gradually transitioning more and more into Rust land.
They often have horrible security, but are very small programs when you use them as wifi temperature sensors or similar. They are good targets for rewrites and greenfield applications. Right now it seems like ardunio C++ or some scripting language is the target.
Rust is a great choice in theory for anything with an LLVM target, but the libcore "bare metal"/"no standard library" ecosystem is pretty immature, and there are likely to be bugs (like the compiler generating binaries that are too large, etc). If you'd like, please try it out and file any issues you find :)
ATS is a functional programming language that does not use a garbage collector and uses dependent and linear types to do safe system programming. It provides a similar level of safety to Rust with the ability to drop down to C: http://www.ats-lang.org/
unfortunately there's no LLVM backend for the xtensa cpu used by the ESP8266. There's a gcc backend though, so if someone has the time and skillset to port to LLVM, here's a way to become a hero ;)
Don't waste your time rewriting almost unmaintainable legacy stuff. Perl actually died because the Perl 6 developers made the mistake to retain compatibility with Perl 5 which was not possible due to its unmaintainability. At first it looked promising but later on it turned out to be impossible. Rust could fall into the same trap by retaining dependency on C/C++.
I would use Rust for a complete startover. Open source development should shift its mentality to support open hardware. This could ensure safety of both hardware and software in the future, and it would attract many developers who want to contribute.
We already have the tools for open hardware: free IP cores, 3D printers, free CAD, equipment for DIY PCB/SMD boards, etc. ... and Rust for bare bone programming.
"[Other safe languages] require complex runtime support that doesn't fit in certain contexts (e.g. kernels)."
Complex runtimes are also unsuitable for libraries. If you write a great openssl replacement in Haskell, it's only useful for Haskell applications. Because nobody wants to link the Haskell runtime into many of the applications that use openssl.
For widespread libraries, the choices are basically C, C++, and now (hopefully) rust.
Which is "for widespread libraries, the choices are basically C, C++, and now (hopefully) rust.": as in C++ you can use high-level constructs internally without relying on an extensive runtime, and expose and easy-to-FFI C interface.
Rust still has insufficiencies though, the inability to customise the (library's) allocator is one I think.
Does the custom allocator support allow for runtime customisation e.g. the library user calling it to set allocator functions then starting to use it? And I expect the custom allocator is global?
Currently, it replaces the global allocator. Using multiple allocators is an interface that's still being actively worked on, it's in the RFC stage, so no code has landed yet.
There's a perverse incentive in software, where toolsets requiring memorization of arcana not inherent to the problem domain but inherent to how the tool maps onto the problem domain, will generate irrational loyalty among its users. Our toolsets and the investments we make in them really do become a way of life, and fuck the world if it tries to get us to change.
1: So a new generation, new rewrite of everything... Then hitting brand new problems (sometimes old ones avoided by previous designs) not foreseen by majority. A new language spawns with the coming of a new generation, addressing some of these problems. Rewrites everything. Hits another set of problems. Goto 1.
I am suggesting that the problem is elsewhere - and more philosophical. I understand for many very young kids learning JavaScript was the only accessible way to do programming. Yet they are producing way worse stuff than previous generations unfortunately (remember Scandinavian demos in the 90s?), as their platform of choice is seriously limited. Now the article mentions horrible things in code developed over decades. I can guarantee you most of decade old code looks similar. And most likely decades old code in Rust will be similar. Here is a need for some simplifying meta-programming theory instead of slightly modified language constructs known since 60s.
So what I see is iteration with some progress and some regress (wrt to empowering programmers and simplicity). It's good to have a new generation enthusiastic creating their own world, yet if they are as human as previous generation, most likely they would hit the same problems. Like a genetic algorithm, a slight mutation to current solution runs for a certain time and we see how the generation fares. But maybe we need completely different approach to get out of local optima technology seems to be right now. Quantum Rust? ;-)
Frankly, Rust just takes a few different approaches known since "forever" and promotes them to 1st class semantic citizens. Similarly Go takes another set. Like every language in existence. So some things will be super easily expressible in one or another language, other things more obfuscated. Yet you could do the same in older ones, but perhaps more verbosely, with more syntactic sugar or boilerplate structures. Hence the comparison to a genetic algorithm with a few mutations in each generation. All a variation of the same set of mental constructs, possibly with very similar descriptive abilities (even when leaving Turing completeness out of the equation).
Rust of course has a standard library[0] and it includes a libc, but (I might be wrong here) I think the libc portion is FFI into glibc or similar. I think the stuff written in rust is considerably more high-level, focusing more on data structures, concurrency, string manipulation, etc.
There is the library 'libc' which is a collection of interfaces to (g)libc functions together with the necessary constants and structs [1].
There is the library 'nix-rust' [2], which aims to provide a wrapper of 'libc' in idomatic Rust.
Unlike 'libc', 'nix-rust' does not have a consistent code quality and organization yet. However, Both are missing various bindings for various platforms.
How dependent is Rust on Mozilla for support now? Has it developed its own community of contributors? I like Mozilla and their mission, but with the way things have been going the past few years I wouldn't want to put a lot of time into porting my projects to something whose future was dependent on Mozilla.
As of just now, we've had 1323 contributors to the main repository, not counting Cargo, some crates, or the website, for example.
Mozilla is paying virtually all of the people who are working on it full-time, but there are a lot of people doing a lot of work in their spare time too.
> with the way things have been going the past few years
If you're referring to the projects which have been dropped over the past few years, I can see that. At the same time, Firefox is the core product, and Rust is already in Firefox, so dropping it doesn't seem likely. The amount will only be increasing.
If a given piece of software has been largely written using certain patterns or conventions, then it is often possible for a syntactic rewrite engine to be able to match those patterns, then output idiomatic code patterns in the target language.
This may well be applicable to kernels and other constituents of an OS.
Have you even looked at the source code to glibc? It's stuffed full of macros and #ifdef hell. Not to mention symbol versioning, the use of gcc-specific compiler extensions, and ancient-UNIX-beard performance hacks. You'd have more success teaching a chimpanzee to play the violin, than automatically translating glibc into anything.
I've seen a lot of weird preprocessor (ab)use in GNU code. The coreutils true/false (of all programs to invoke clever preprocessor tricks...) has this gem:
in true.c (all told, an 80-line file):
/* Act like "true" by default; false.c overrides this. */
#ifndef EXIT_STATUS
# define EXIT_STATUS EXIT_SUCCESS
#endif
But BSD true doesn't support --version, how are you going to check your version of true without a locale-aware version string (and yes, gnu true is locale-aware, and contains 2k of strings including 48 encoding names and 36 format strings — 10 of which are duplicates, and for some reason refers to the SHA2 utilities)
(I dimly remember an essay about similar issue in a similar mainframe command (maybe in JCL?) taking a bunch of revisions to actually get right?)
So I guess we need more than just a "transliteration" to a safer language. We really need to shed that cruft and write something clear and understandable.
Could we also drop a lot of requirements that applied to older, more limited hardware? (Or, do certain constrained hardware situations still require most of those requirements to stick around?)
It does seem to me though that if you pre-processed it for a common case and translated it to rust with anything going into unsafe if it has to and started from that you would be off to some sort of start.
The whole thing seems moot though because there are already alternate libc implementations like musl as well as alternate malloc implementations like jemalloc.
I actually still don't understand why libc is the undertaking it is made out to be, especially if you can plug in jemalloc which I would think would take care of the most difficult part.
I realize something like glibc would be the LAST set of source code to run through a transpiler, but how hard do folks think it would be to make a C -> Rust transpiler? I'd think you'd go about it by making Rust a target backend for LLVM. This way you let LLVM deal with the all C stuff (macros, etc), and just focus on the internal representation to Rust conversion. In theory, you'd get C++ to Rust as well.
Chances is successfully going from some not-so-obfuscated C code to equally not-so-obfuscated Rust code?
> how hard do folks think it would be to make a C -> Rust transpiler?
Why do you want to do that? The only way to make it work short of doing years and years of static analysis research to automatically translate ad-hoc memory management disciplines into those of Rust (that I think will not succeed anyway) would be to compile to unsafe code. And if you compiled to unsafe Rust code, there would be no gain.
I'm not sure it would take "years and years" of static analysis research. You could use the data-flow analysis that is built into LLVM, then punt on the rest. It doesn't HAVE to produce Rust code that compiles 100% of the time. The goal would be to run it through the transpiler so you have a starting place to produce Rust code that compiles and works the same as the original C code.
Again, this might be a terrible idea, but I'd be curious to hear from anyone that has experience with this sort of porting...
> hard do folks think it would be to make a C -> Rust transpiler
Easy. But it won't be too useful.
Rust isn't a compiler that magically finds memory errors. Transpiling C code to Rust will not find those memory errors unless you have a really good transpiler that can detect the overall structure and design of the C code. It will just give you a ton of Rust error messages; probably none of which relate to an actual memory error. (Or you transpile to `unsafe` rust, in which case, what was the point?).
Rust works with a set of rules, and these rules enforce some things in your programming/software design style. This enforced style (i.e. the discipline around memory usage) is what gets us memory safety. This style is not the same as the style used in programming C/C++; so a dumb transpilation won't work. You need to write a transpiler that can figure out what logic the code (and not just for a function, for a larger unit) was trying to encode, and write that in Rust. That's ... a nontrivial problem to solve with a transpiler, probably requiring AI skills. It's a mostly trivial thing to do as a human, since mostly such "transpilation" involves noticing a few key issues (related to highly-shared structs and whatnot) and fixing them (leaving most of the code the same).
Generally the opinion is "it might work but it would be terrible Rust," and given that FFI into C has no overhead, replacing things a symbol at a time might be a better approach to this kind of problem.
A compiler has more than syntax though, since it understands semantics and does a lot of checks. I think the parent comment refers to something a little more rudimentary, like a macro-based system to do a rough translation of code.
IMO -- Better is to just write better tools for plain olde ANSI-C. We can build tools to check C as well as any other. Let's just not waste time on building further complexity into our toolset and look at building toolkits and best practice at the very flexible level of abstraction that Plain olde C provides. With the right tools and expertise, we can improve the world without the never ending whiz bang rewrite.
> We can build tools to check C as well as any other.
This is simply not true; language semantics determine the kinds of tooling you can build. Ruby is so dynamic that it's notoriously hard to build tools for, for example.
Doesn't modern C++, with smart pointers, auto, and the rest, cover most of the same ground rust does, and isn't the Rust community somewhat toxic, with activists pushing "Codes of Conduct" and other non-technical agendas?
>We live in a C world and I don't see that changing any time soon.
Of course it won't because every time an out-of-bounds array access gives attackers free reign of a system and some one says "hey guys this whole C thing is clearly blatantly terrible" every one else says "Not this crap again. C libraries are not pretty but they have been out there for decades, they have been reviewed and used in production."
As if the libraries have been utterly static for decades and aren't infact full of holes and bugs, which is exactly what brings on the call to rewrite things in Ada/Rust/SML/Lisp
Then they always say "There is no silver bullet, just because you use Rust it doesn't mean your programs will be completely safe." As if there's no reason trying to improve things unless you can jump directly to perfect.
they make excuses like "A lot of these C libraries were written in more innocent times where a small bug would not affect as many people as it would today." as if we still lived in those innocent times.
And then they inevitably wave the call to action off by appealing to the status quo with some line like "We live in a C world and I don't see that changing any time soon." and so even though there's no reason we couldn't start re-writing things (especially in Unix an OS that constantly brags about being built from loosely coupled parts) in a better language we never get a critical mass together.
My real gripe with C is that it was designed to be insecure ON PURPOSE, and many people in the OSS world hold the religious view that this is a good thing.
C's primary design goal that overrode everything else was easy implementation. In order to keep a dumb compiler, the preprocessor was used instead of a proper module system. For the same reasons, the standard library is full of non-reentrant functions relying on global state, unbounded string manipulation functions or bounded string functions with ambiguous off-by-one properties, vaguely defined data types and so on.
If there's one reason to do away with C, is to do away with all the stubborn programmers who clutch their copy of K&R and think everything must be done in the "C way". To paraphrase Linus' on C: you'd want to avoid C just to avoid this type of C programmers. They could be amazing coders, but not if you care about security (and you should).
Just gonna throw this out there: what if instead of working on creating the myriad of "better languages" we focus on actual mastery of simpler languages like C.
And with that mastery: what about writing some machine-learning software that detects vulnerabilities as part of the build/release cycle. For instance:
To detect vulnerabilities people need to be careful and follow best practices. If they can't do that well.. replace the people with software that's better and more efficient than the people. :)
we focus on actual mastery of simpler languages like C
We've had 44 years for people to master the "simplicity" of C. Turns out, writing C is simple but writing safe, reliable C borders on the impossible even for the most masterful masters who ever mastered their mastery of the language. Better return on investment would be to work on killing C with something that doesn't have C's problems.
>Just gonna throw this out there: what if instead of working on creating the myriad of "better languages" we focus on actual mastery of simpler languages like C.
Relying on people being careful does not work for security, where one exploit is as good as many. OTOH, simpler languages can work well for performance, features and almost everywhere else; where even if you don't have specific language features to guard against common mistakes, the impact of an oversight is fairly localized or correctable.
Why can't we do both? On the one hand, we obviously need better techniques to maintain and improve upon code bases in languages like C. They aren't going anywhere anytime soon. But why shouldn't we also pursue improvement to the programming language state of the art? And if we find improvements, why shouldn't we use them? I'd be very sad if 50 years from now C (or Rust, for that matter) is considered the final word in programming languages.
Off topic, but it's surprisingly refreshing to see someone put "Christian" next to their name in the same place they put "Mozilla hacker." The vast majority of programmers either aren't religious or hide it so well that you couldn't possibly tell if they were. (Not that being overt about it is better, either.) I understand that it's generally best not to talk religion in the professional workplace, but a person's own blog is a more neutral place to bring it up, so it's nice to see that here. It's also refreshing to see that there are other adult converts to Christianity in the programming world, as opposed to from it, which seems to be the norm in general but particularly in this field.
Being a Christian myself, yes, I do find it refreshing specifically because he's another Christian programmer. It's very rare that someone is both, let alone an adult convert to Christianity.
Rewriting the software in Rust would not solve these problems any more than rewriting it in C++ would. A rewrite would clean up the code, sure, but then you're left in the same situation, only with brand new bugs that nobody has time to fix.
We need to have incentives for maintaining this foundational software.