I have a colleague who had a dedicated PC that had to be on at all times. The company bought a UPS for it, and it was on patch exception lists to prevent automatic reboot.
One day, I asked him what it was and he showed me. He was running a series of excel macros that took over 96 hours to run, and fed a critical business process that was worth a few million dollars.
I sat down and took a look at it, and in about 3 days implemented a prototype in SQLite and a small Perl script that ran in about 8 minutes. Basically, it was a series of sql joins and some regex. My colleague, a project manager, literally hugged me and was near tears.
Some devs later took that prototype and refactored it to run on a normal production platform.
This is interesting at the bottom of the readme file
Jez also offered this interesting BRender anecdote in an email:
When Sam Littlewood designed BRender, he didn’t write the code. And then document it.
The way most things were built at the time.
First, he wrote the manual. The full documentation
That served as the spec. Then the coding started.
I haven't come up with a theory beyond Twitter hates tech communities but that feels off..
Might simply be "money". The "Web 2.0" business model, which actually sometimes makes money, is Revenue = Transactions per second X Revenue per Transaction - (Capital Expense + Operating Expense). It really is that simple.
Capital expense is both the cost of servers and what not which records quarterly as depreciation expense, operating expense is everything you pay per month, whether it is developer salaries, licensing fees, or network bandwidth charges. And transaction revenue is typically in units of $ (or local currency) per 1,000 transactions.
Twitter, is an information business. That is to say the value it provides to people who use it are the consumption of, or distribution of, information.
In previous technologies, one could inject "forced information distribution" (aka advertising) into the "good information" that people valued. And for MBA types without any vision that became their "go to" mantra. We can force ads on the users which people will pay us to do.
It is an adversarial relationship to be sure, there isn't much incentive for Twitter to be "compelling" like there is for audience metric based information services like television shows or movies. And there are plenty of technological mechanisms to utilize for injecting information and for collecting demographic data and reselling it.
The failure in that model, and the one that probably shut down the event tweet network, is that Twitter was not getting enough money from NYC_TechEvents and others for their use of the system in these ways. The "easy"[1] coin was personal data and forced impressions. As a result anything that interfered with those revenue streams was to be shut down. The road not taken was to exploit the time value of information and the imbalance between readers and writers.
There are a number of ways that information gains value, one is its timeliness. The canonical reference for this is stock market quotations. Getting a stream quotes that are 15 minutes delayed is "free", getting those same quotes within a few seconds costs money, and getting them within milliseconds costs exponentially more money. Notice that the "information" here is all the same, it is just "when" (or the time) you see it that is valuable. So let's apply a business model that collects the value in "timeliness" of Twitter.
Twitter keeps me up to date
To extract this value, create two tiers of service, one which is free and the Tweet stream is 15 minutes delayed, one which costs $4.99 / month and the tweet stream is instantaneous.
Twitter lets me reach my group
To extract this value, create two tiers of service. One is free and reaches 10 people per minute. Which is to say your tweet goes out and 10 random followers see it right away, and then in the next minute another 10 see it, and in the next minute another 10, until all of your followers see it. A paid account can increase this diffusion rate from 10 to 100 @$4.99/month, to a million $49.99/month.
These are just a couple of examples of creating a model where the more you value timeliness of the information, the more you have to pay for that. What this does is provide a system that captures the intrinsic value of timely information and allows you to create a value proposition around it. No advertising, just people talking to people, and some wanting to talk to bigger crowds or stay more informed etc, and a whole lot of people who are fine with things being delayed a bit because its free.
Its fun to model these things too. You can analyze what sort of return you would need to have a margin that sustains a business and it becomes just like modelling making widgets in a factory, all of the classic business mechanics work like you would expect. Pricing the value add is just as hard as it is in a goods economy, and pricing mistakes are just as costly. But it equalizes out exactly like it does for goods.
[1] The term easy here is a way of expressing a technique that has worked, for the definition "generated revenue" for worked, in other situations.
What's liberating about talking with strangers is they don't know who you are, where you came from, what you've done, what you haven't done, who you know, or what you're like.
You can say whatever you say knowing you can't be judged by or held to some constraint from the past.
If your views on euthanasia are changing, because you suddenly have a close elderly relative or an associate whose circumstances are changing, a stranger won't berate you for having attended church for most of your life, even though you advocated the church's ideals which included no support for euthanasia. You can just talk about that subject in the context of your current circumstances, needs, and beliefs.
That's liberating and likely to be informative.
If you want to try out or finesse a different political point of view, such as that capital gains should be taxed more or less, or the death penalty is immoral, or social security should be limited to people who have worked and paid taxes for a minimum of 7 years, you can try that with a stranger and just listen to what they say, without having to assure them that you are from a friendly tribe or that you have an open mind, because they won't know your income or your status or your voting record.
When you speak points of view in an anonymous guise, or at least, one that isn't tethered to any prior baggage, you gain an opportunity to refresh your understanding in ways that you normally don't get from people who have such well developed expectations about you that they can't really hear what you are saying.
The same is true in deeply personal matters such as affairs of the heart, and confessions of moral confusion and even complicity in crimes. For many people, it's not until they hear someone's completely independent echo of a picture they've painted of themself that they begin to see themselves in a different light.
The great thing about this process is it isn't limited to one stranger, in the same way that your conversations with your bff is limited to only your bff.
Strangers are everywhere, and once you know how to strike up a meaningful conversation with them, you can do it again and again.
If you've never had this kind of conversation with a stranger, how can you be sure you know who you are?
You have to connect thousands of cables, have hundreds of nodes, with thousands of components, everything interconnected, and if you connect one wrong, the computer outputs incorrect results. You have to routinely update the software, and if a software upgrade introduces a 20% perf regression (which happens), then your 10 MWh cluster starts burning 2MWh for nothing. Or maybe your cooling system sucks, and after a minute of running at full capacity, you need to throttle your cluster to 0.1% of the peak to keep it cool enough that it runs "something".
That's why all systems in the top500 i've been involved with (15 or so) run these benchmarks as an integration tests on every single cluster maintenance (node updates, servicing, OS updates, etc.).
Submitting these results to the Top500 costs you nothing... if your cluster actually works. When you submit to the Top500, they ask for access so that they can re-run them themselves, which happens typically during / after the next maintenance to avoid impacting any users.
If they haven't submitted, 100% sure their cluster does not deliver what they say it should deliver on paper. Maybe it delivers 1% of it, or 0.01% of it (seen both cases in real life). If they haven't fixed it, then maybe it can't be fixed.
HPL, MLPerf, Spec, Stream, OSU.... these are not "benchmarks for bragging rights", these are tests that show that your system works.
I've been using Tiddly Wiki for years, and can very much recommend it.
To start, all you need is a template HTML file. No installation, no nothing - it's self-hosted within the HTML! You save changes by clicking the "Save" button, which generates an HTML with changes.
If you really want, you can install a Node.js version which saves changes automatically. But you don't need it to start.
An interesting thing about Factorio to me (beyond what’s stated in the linked article), is that it contains a nearly a perfect 1:1 analogy to software concurrency.
• Belts are blocking CSP channels, as seen in e.g. Golang (where N producers have to share — or eventually “merge onto” — one channel representing the blocking-receive point of the consumer; which are best only used to transport messages of a single type, or if not, then the sum-typed channel must be demultiplexed at the end, with this tending to lower throughput; where messages of a given type “queue up” until the channel is full, and then all producers block when attempting to write to the channel; where if you’ve got producers producing at different rates, then the fastest ones can hog the capacity of the channel, decreasing throughput and unevenly spending input resources such that some components fail long before others; where the solution to this is to give each producer its own bounded outbox channel that multiplexes onto the consumer’s channel, such that the producer will block itself rather than blocking its siblings; etc.)
• Logistics robots are message-queue topics (where N producers can publish messages of a given type, without worrying about how they’ll get to a consumer; where consumers [demand chests] subscribe to specific event message types; where the bus itself can get overloaded, causing delivery failures of unrelated topics as delivery-threads sit around holding a message unable to deliver it; where the solution to this is to add reliable MQ-internal storage [network-linked storage chests] for the agents to offload produced messages to until demand comes for them.)
(Sadly there’s no exact equivalent to Erlang-style message-passing, where producers target messages of arbitrary type at a specific consumer, which all go to the consumer’s single inbox; and where, if that inbox is full, then the message just “evaporates” en route, since the passed message has async semantics where the producer isn’t linked to its delivery. But, interestingly, that type of concurrency totally could be added, with a not-even-very-complex mod — just add a “outbox” chest object that can be configured to “target” a specific “inbox” chest somewhere else; and a second type of logistics robot that only moves stuff from outbox chests to inbox chests, not according to “demand” but just because anything currently sitting in an outbox chest is “intended to be” in the corresponding targeted inbox chest; and then ensure that this alternate type of logistics robots have non-reliable delivery semantics, where if the “inbox” chest signals to the network that it’s full, then all active delivery-threads targeting that inbox will literally “drop their messages on the ground”.)
IMHO it’s actually possible to learn how to be an effective distributed-systems engineer, just by playing Factorio and trying to scale the throughput of a resource-constrained system. In the process, you’ll likely re-invent many real-world concurrent software design patterns. Doing this first, and then reading a Distributed Systems textbook, will have a much more visceral impact on you, because you’ll have already faced these problems, struggled with them, and now you’re being handed all the techniques for solving them on a silver platter.
