Frankly, I believe that the criteria for the language selection for throw-away code is just that:
> This certainly is in part because I’m more proficient in Rust than in Python. It’s also because the Rust mental model is closer to how my brain works than the Python one.
You'll be more productive in writing the quick dirty hacks in language and programming model that you're the most comfortable with. That's it, the rest is IMHO just an attempt to rationalize your intuitive feelings.
Something I realized recently is that when I'm hacking on something, I like to brainstorm in terms of data from the very beginning, which means brainstorming in terms of types. Static types aren't just about stability and error-catching, for me. They are their own expressive vocabulary and I feel expressively crippled without them, even when working on a rough draft that will likely get thrown away.
That said I tend to prefer TypeScript over Rust for fast iteration, all else being equal, but I agree with the author that Rust can work quite well even for this usecase. And I would never pick Python for anything beyond 100 lines, if I'm being honest.
If you haven't, you should explore domain-driven design (DDD). It's an explicit idea in that framework that a problem needs a "ubiquitous language" and an identified "bounded context" where the said language is valid.
Like you, when approaching a new problem I like to nail down the terms we are discussing, literally scaffolding classes and types (in C++) as the problem is being discussed. It drives the adoption of unambiguous terms. Then as solutions are discussed the "ubiquitous language" helps clearly identify the interactions between types.
One advantage of using a very open-ended type system like TypeScript's is that you don't have to know as much up-front.
I could declare some type Foo, and that type might end up being:
- A class
- A record type
- A union type
- An intersection type
- A constant
or whatever else. But I can go ahead and start referring to that type in terms of function signatures, etc. Particularly for this kind of sketch-coding, it's really nice to be able to declare a concept and start talking about it before you even know what it is.
Scala also shines for this style of development. Sealed traits and case classes (algebraic data types) are a nice way to express many different types of data models, including polymorphic types and state machines (its surprising how many "hidden" state machines live in your objects via Boolean and Enum fields).
I found this book very helpful in terms of thinking in a more functional type-driven way: https://www.manning.com/books/functional-and-reactive-domain.... It does eventually get into some deeper functional programming concepts but its still a nice read and easy to get into for anyone coming from more traditional object oriented coding.
For sure, the compilation times can get pretty bad with Scala, although I haven't found it too terrible on smaller prototype projects (I agree its still quite slow compared to some other languages though).
These day's I've been using Bloop (https://scalacenter.github.io/bloop/) which makes Scala compilation super fast. It keeps a hot compiler running in a daemon and takes incremental compilation to a whole new level for Scala, truly a game changer for me.
100% this. I’d go further and say that ts lets you write pseudo code that compiles. I will often mock up all the inputs and outputs for a system without writing any code in a shareable ts playground. It’s like an extraordinarily expressive design doc. If only management could read it...
> I like to brainstorm in terms of data from the very beginning, which means brainstorming in terms of types.
It's funny, because I feel the same way, except I gravitate towards Clojure, where data is just data (and you stick it in vectors and hashmaps), and I don't feel the need for static types. At a later point, I may think about adding specs, but for quickly iterating at the REPL while I think about a problem, "plain data" works wonders.
I think the difference is that with types, you have a language based reference which you can talk about (fn x takes type y). This allows you to build the lower functions and check the abstractions at compile time, without having to build the higher levels or even run the program.
With a dynamic language, you need a specific instance of data in order to write your functions, so you typically start from the top function and implement downwards. I think this works well when you have a light layer over some other library or system, but does not work well when you are implementing many layers in your own code.
I think this is what he hints at with:
> When I want to know if my code is working, I actually have to run the Python thing and feed it with data
I like REPLs, but Im experimenting with a type system + running a test on save that exercises the functions Im currently working on. This feels REPL-like as the iteration loop is fast.
It's interesting because I would previously implement something top down from the highest functions, but now I can implement mostly bottom up as the types/compiler gives me the language to define the layers.
For what it's worth, my usual flow is to start sketching top-down, and building bottom-up afterwards, refactoring as I go. I also (ab)use Clojure's argument destructuring and will often just pass a map until I settle on the "correct" API for my code.
Not bashing good types (I do like Haskell, too), just showing an alternative based on the same brainstorming idea the GP had mentioned.
This is big! I call it (others might too, idk) “.d.ts driven development”. Where you start the project by writing a .d.ts file and implement it. Similar to starting with a .h in C. Fun fact this is how xterm.js is structured (the terminal emulator used in vscode and hyper, among other things).
Yeah, there's this notion that with static typing, you have to "get your types right first". In my experience, when I'm prototyping it's often very effective to sketch out my current understanding in the types, and then as I learn - quickly and slowly - where I was mistaken and update the types, the type checker helps me update the code that needs updating to match.
If the process is rather easy (fast to implement) usually I do the opposite. I implement the process with everything as "any", and when it's working I begin to define the types and fine tune it.
I could not imagine choosing Rust for the use case described. I think it's one of the worst possible choices!
Python has so many advantages for throwaway code with quick iteration. Duck typing. Easy string indexing. Bignum arithmetic by default. Trivial mutable globals. Default parameters. Trivial to make reference cycles. Access variables by name. Exceptions. And the big one is eval().
That's what makes the article noteworthy. There are a thousand articles of various quality out there telling you some variation on "dynamic languages are useful for prototyping and scripting, Rust/C/Java really shines in large codebases / when performance or reliability or security is important".
It didn't convince me, though. There were a lot of "this tool or library or feature exists in Rust", but no comparative advantage over other languages more compelling than this:
> This certainly is in part because I’m more proficient in Rust than in Python. It’s also because the Rust mental model is closer to how my brain works than the Python one.
Also I really like Python stdlib for those kinda of throwaway scripts. While `cargo` is a far better tool than `pip` for dealing with dependencies, if you don't need to dependencies at all it is even easier, and most times when using Python I don't really need to install any third party libraries even when doing non-trivial stuff (i.e.: sometime ago I did a script that scrapped a page for a download link for some APK and sideloaded it to my Android smartphone).
Someone also said the REPL, and while the REPL in Python is not as powerful as, i.e.: Clojure, it is still something that feels essential for a exploratory workflow that is the scripts case. Testing is not the same, it is much slower to interact, specially when you just want to check a hypothesis (also, most scripts I don't want to write tests, they will probably break anyway in a few months like the example of web scrapping I said above).
> Someone also said the REPL, and while the REPL in Python is not as powerful as, i.e.: Clojure, it is still something that feels essential for a exploratory workflow that is the scripts case.
Check out IPython if you haven't, the REPL is a game changer.
What features make it a game changer for you? For me it felt like just a small quality of life improvement, but I've never delved that deeply into its feature set.
I don't want to disparage the pragmatic value of the features on your list, but they all pale in comparison to the one true killer feature for fast development cycles: a REPL.
> the one true killer feature for fast development cycles: a REPL.
I've been a python dev for about a decade.
I do use the REPL a couple times a week, but I wouldn't really call it "essential".
How do people use it that makes it so valuable?
Usually I just use it when trying to figure out how to parse some text or check what methods or properties are available on an object. Really nothing that I couldn't do in a test suite or the code itself.
If you're not writing your code in a design that lets you run parts of it every few minutes (or more frequently), you're missing out. Best way to avoid bugs in your module: don't put them in there. Find out in the REPL before you put code in the .py file.
Also, it's more important the more data-dependent your code is.
Tests also have the advantage of input automation. They have the disadvantage that you can't poke around at the environment once they're done (unless you drop into a debugger).
I usually have a test data set and run my module against it in IPython. So input is automated to some extent. If I'm analyzing the output, it can be helpful to quickly adjust charts and graphs. I'm now in the process of building distribution tests into my test suite, but it's not clear how to track everything. I need data sources essentially kept under version control, but which are generally too big for git.
My personal choices are IPython and Jupyter, depending on whether I'm making plots. If you choose Jupyter, though, I caution that you should be careful you don't switch entirely away from modules and test suites. Jupyter can be too easy in a way.
This sounds quite a bit different from the sort of development work I do.
I can see how a REPL would be really useful when working with data, but for building out a webserver or desktop application, it doesn't have nearly as much value.
And tests have less value, or at least are more difficult to write, when you're not sure if an outlier is valuable information or junk to be discarded.
I'd love to see a video of someone developing via the REPL that somehow let me "get it".
I've never written code for which I've found the REPL that useful so either my domain is not conducive to REPL usage or I need a good tutorial or example to see what I missing and how to change my approach.
Any suggested videos or articles? though i suspect a video would be better for this type of thing
Funny, that. I see the REPL as a trap. (IDEs, too, for other reasons.) REPLs are a trap because they give you the illusion of progress while mostly just wasting your time.
I am used to writing 2000 lines of code and having it run (mostly) correctly the first time. In C++, in my case. Otherwise my experience is like the author's. Types are not mainly for error checking. I demand they earn their keep as part of the solution, and they do.
It depends a lot on what the program is doing. Writing 2000 lines of C++ that interface with other C++ libraries and having it work first time is something I can understand.
But if you have to consume untyped input (such as parsing text), or produce untyped output (such as an image) then the value of a Notebook/REPL really shines.
eval is useful for making a repl, but there are languages with a repl and no eval (Haskell comes to mind), and there are languages like PHP that have eval but no (built-in) repl.
Hmmm, python is not installed by default on Windows. Python3 is not installed by default on MacOS. Installing them is 30+meg and now your system is "infected" with python3 (I know if you use python every day this isn't an issue to you but as someone that's not a python expert and has run into version conflicts it's no fun).
So, AFAIK rust will generate a standalone exe, python, out of the box, will not so I want to give someone else my throw away solution, not requiring them to install python to use it might be a win?
Good type systems don't get in your way. I have often built prototypes in Scala and I felt the type system was speeding me up rather than slowing me down.
It can be nice when you're working running code in a repl and don't want to maintain state/code between repl sessions. If you just print the data to stdout, you can just eval it and not have to deal with writing one-off cacheing. It's similar to the use case of pdb where you're forcing yourself into the middle of ten different functions. But print statements are easily added, easily removed, and so is eval.
> Over the years, I’ve used Perl a lot (that one doesn’t care if it’s int or string… no, correction, in Perl everything is a string, ints just don’t exist. Well, kind of). It’s probably the language designed for throw away coding. I’ve done some Python too (that’s like Perl, but with proper objects in it, and everything is a dictionary there).
The author has almost no real programming experience with Python. Perl experience seems to overlap with sysadmin-related work at least partially, where it's usually used as better Bash. All their repositories in GitHub are Rust. So almost all of "real" programming they did used Rust.
Why would anyone who know Rust way better than any other language prefer to do their prototypes using anything else?
And the claim about how Perl behaves and is to be programmed in is totally, completely wrong.
For the start, one couldn't even write a correct "if" condition in Perl without knowing if it's about numbers or strings. Even if the comparison is over the "scalars."
Second, the numbers not only exist, one can control which numbers are used when (one can explicitly force integer underlying types for some expressions or code blocks, or keep the floating point calculation, or even turn on infinite integers).
Third, Perl has scalars, arrays, hashes, references, which have even different symbols throughout:
so the types are more obvious when reading the language lines, much more than any language where every use of the variable looks almost the same.
That's why it looks "too much like line noise" to those who don't know the meaning of the symbols. But having these symbols makes the code somehow "firmer" -- the programmer must much more often write what is expected from some variable, and the expected behavior is more obvious when reading.
I don't know if I entirely agree with this, but I do think that at the very least Rust is much better for this sort of development then some would have you think.
It's strict, yes, but most of that strictness helps with eliminating the types of errors that make your app break over and over again at runtime. Unit tests can help bridge the gap, but who wants to write unit tests for throwaway scripts?
When you get to know Rust, it does seem to have plenty of little bits of polish to help you get things going fast. Most of the stuff that the compiler/borrow checker complains about is either easy to fix even if by throwing around clones that might not be the best performing, or represents real issues that are better to think about now than trip over halfway through some processing run.
> It's strict, yes, but most of that strictness helps with eliminating the types of errors that make your app break over and over again at runtime.
Really? Most throwaway-script errors I get are simple things like IO mistakes, mistyped URL, or logic errors. The rust borrow checker is not designed to catch simple mistakes, it's designed for massive multi-threaded applications, e.g. making sure threads access data safely, making sure a variable doesn't get unexpectedly modified by some other file, making sure memory is owned by exactly one place, etc. But in a single-threaded garbage collected script that can fit on your monitor, why should you care? The borrow checker does not check for logic errors; it checks for the things you don't care about.
Those kinds of errors are made trivial to handle with ? and anyhow. At least, once you’ve got a bit of practice with it. And it’s barely slower to write out; 1 character in most cases, and “Result<>” on your return type.
use anyhow::Result;
use chrono::NaiveDateTime;
use serde::Deserialize;
use serde_json;
#[derive(Deserialize)]
struct Person {
name: String,
age: u32,
hired: NaiveDateTime,
emails: Vec<String>,
}
fn main() -> Result<()> {
let data = std::fs::read_to_string("agenda.json")?;
let mut people: Vec<Person> = serde_json::from_str(&data)?;
people.sort_by(|a, b| b.name.cmp(&a.name));
for person in &people {
println!("{} ({}) - {}", person.name, person.age, person.hired);
for email in &person.emails {
println!(" - {}", email);
}
}
Ok(())
}
I learned that Chrono supports serde so I can remove my own parsing, and using ? instead of unwrap, because it's not harder and it ends up being nicer overall.
(EDIT: Ah actually, it won't recognize this specific format automatically, so I still would write the parsing code. Even then, it's still roughly in the same ballpark.)
It's not really about the borrow checker, but the rest of the Rust type system. Even with throwaway code that can help catch some problems. Is it worth the tradeoffs? Unclear.
I love writing throwaway code in Python. It is really great for that and there is something about Python and the hassle you have to go through to write safe, well-structured code that makes you wanna say screw it and hack it in there.
I've been coming at this from the other side and writing typed Python for my little experiments. You can add just enough type hints to make life easier, but not go obsessive over it. Then vim with Python ale tells me if I've done the right thing in a deeply nested loop without having to actually execute it.
I've really embraced type annotation and mypy for Python. Partly because it really helps the Python language server in VS Code with autocomplete. It has also saved me a few times in pointing out some type mismatches and forcing me to really think about what I'm returning, etc. It also makes reading the code easier in terms of reasoning about it.
I've noticed this similarly switching to using typescript over javascript. I'm not a huge JS dev anyway so I don't use it much, but having type signatures to enforce requirements, but being able to turn them off when they get in the way is great for productivity. Of course in a productive environment I would probably enforce strictness of the types but for my own terrible front ends, I don't care
> It also makes reading the code easier in terms of reasoning about it.
For me that's huge. I work in a support role where I pretty much exclusively look after code other people have written. Without type hints when something blows up I need to follow the code all the way back to whenever the variables were initially substantiated so I can know what to expect and what I can do with it. If I have a type I know what I'm looking immediately and start building a model of things without back tracking.
I've been doing this for years, and can't go back. The typing module is pretty comprehensive now, and using pyls-mypy with an editor like Kate that has an LSP client means you can type check as you're writing Python code in your editor.
I'm not sure if the author really meant this phrase the way it sounds, but if he did, it's a bit worrying:
> That’s one of the points of Rust. It’ll complain that our code is not production quality and that we need to do better to save on the pain down the line.
Ok, Rust will catch a lot of bugs, but assuming that a program is "production quality" once the Rust compiler doesn't complain about it anymore would be really bad - it may still be full of business logic errors...
The key argument seems to be "type checking and red squiggly underlines save you time" - I would also attest to that, but isn't there a way to do this in Python now? I would also suggest C#, with its Code Completion, as great at helping you write quick scripts, especially now that it has a REPL with full Code Completion, red-squiggly underlines and syntax highlighting in Visual Studio, plus language features like pattern matching, tuples and LINQ.
The C# REPL's not exactly great though. There's a lag the first time you use it and I regularly forget when you need to put a ; to end a statement or not.
I'm fairly experienced and I still can't throw up an HTTP server in Rust, even with libraries.
I almost can do in with NodeJS and Express from memory.
Something like
app = require('express')
app.listen('/', function ( req, res ){ res.send (' Hi Hn') }) ;
It's ok for different languages to do different things.
I can't imagine anyone being faster with Rust than Python or JS. Of course eventually Rust gets a performance boost, but it's never going to matter unless you're talking about a larger scale project.
I would complaint about Rust as well, but setup a HTTP server in Rust is actually not that hard if you accept the fact that you'll have to use third-party framework. Then, the HTTP server will come to alive in within 20 lines of code.
But, of course, the demo code is always easy. The real nightmare starts as soon as your application became complex enough, that the demo no longer fit and you need to manually spawn something (To handle TCP connection before it hits the framework for example) rather than just "actix_web::main" or "tokio::main".
Maybe it's been fixed , but when I tired Rocket last year I couldn't get it to really work.
I would like Flutter to take over since it combines the best of Java /C# ( types) with the best of JavaScript/ Python ( dynamic variables which can be any type ).
I spent a few hours reading the book, but it didn't sink in. I will be honest, though: Rust is still very challenging for me. The learning curve is extremely steep. Perhaps it's because Rust makes some things easy, at the expense of what automatic memory management and runtime type metadata make easy?
The last time I prototyped something in python, we ran into issues with gevent while crafting up a multi-tenet mail transfer agent. Spent a couple of days banging heads against it. Then someone on the team did it in Go in like half a day. The language you choose for prototyping depends on the thing you are validating. I'm also partial to prototyping in the language that the real thing will be written in. At the time, we were not a Go shop, but we are now.
I previously used NodeJS for this, but I missed the system integration you get from python/bash (eg. taking screenshots or playing sounds).
Slowly I'm getting faster in Rust (it does require a particular mental model). What really was the gold in this article for me was the useful crates listed. Thank you a lot for that!
To me it's only throw away code if you actually throw it away after using once (or a couple times). If you are checking it into source control (i.e. anyone else will see it) then it's good to assume that it will live forever. I've been burned too many time when my "throwaway code" or "temporary glue script" becomes mission critical infrastructure
Over the years I’ve learned few things are more permanent than a temporary solution.
This is something your hear in a lot of places. Often (not always) it seems to be spoken with regret. I think we should be quicker to celebrate than a problem has been solved (and seems to be staying solved!) without worrying so much about whether it was done "properly".
Indeed, I think the frequent success of temporary solutions should be seen as a challenge to the prevailing orthodoxy that software developers should be striving towards more discipline and process. Instead, we should be looking seriously at why quick fixes and cowboy coding work so well, and asking whether we can apply these more widely.
In my experience, it is not generally a matter of putting some quick and dirty script in the solve a problem and it goes on solving the problem without modification indefinitely. It is more often:
1. Put a quick and dirty script in to quickly solve some immediate problem
2. When that quick and dirty script stops working, instead of putting a proper solution in place you either add another quick and dirty script in to plug the issue or you modify the original quick and dirty script in some ad-hoc way
3. Repeat over and over again
And then you end up with some impossibly complex Rube Goldberg machine.
I'd certainly agree that if a quick fix starts to accrete complexity, that's good evidence that it's time for a rethink. "ad-hoc modifications", I'm less certain about. How often is this happening? How much effort do the modifications take? Would the modifications still be needed given a "done properly" version (my experience: often yes). Would they be substantially easier?
It can go either way, and I'm certain not trying to argue that quick fixes should never be replaced -- just that trade-offs should be considered.
> This certainly is in part because I’m more proficient in Rust than in Python. It’s also because the Rust mental model is closer to how my brain works than the Python one.
You'll be more productive in writing the quick dirty hacks in language and programming model that you're the most comfortable with. That's it, the rest is IMHO just an attempt to rationalize your intuitive feelings.