"When you have a problem, build two solutions - a deep Bayesian transformer running on multicloud Kubernetes and a SQL query built on a stack of egregiously oversimplifying assumptions. Put one on your resume, the other in production. Everyone goes home happy."
This reminds me of an experience I had watching a company trying to replace a system with ML.
First they marketed it heavily before even thinking. During test cycle they fed the entire data corpus in and ran some of the original test cases and found some business destroying results pop out. The entire system ended up a verbatim port of the VB6 crap which was a verbatim port of the original AS400 crap that actually worked.
The marketing to this day says it’s ML based and everyone buys into the hype. It’s not. It was a complete failure. But the original system has 30 years of human experience codified in it.
The AI taxonomy includes the term "Expert Systems" for these kinds of things. On the one hand it's definitely not of the new wave of ML AI so hyping those things as innovative is off. On the other hand we should definitely give more attention to that kind of setup and understand how to build/maintain/test it properly. Otherwise often it ends up being ran by a few hundred Excel sheets and a few severely underpaid people and that's a disaster waiting to happen. The AS400->VB6->NewShiny path actually sounds like a success case given the messes that are out there.
Often the biggest benefit is that the ML version is good at catching when the experts hadn't had their cup of coffee as well.
Most experts are like family doctors, they get the correct diagnosis 70% of the time. And even if you juice them up real good, they will ALWAYS lose 5% to human error.
The ML also hits the 70% mark, but it's a different 70%, so it'll fix 70% of the errors. Then you're batting at 0.91 instead of 0.70.
If I had a nickel for every time I've seen "business rules engine" turned into "AI" in the last few years...
But I guess if we complain that half of our colleagues and the media don't understand ML, why should we expect management to?
When the command from C-level is "We need some AI projects to tell our shareholders about," we shouldn't be surprised when middle management suddenly has successful AI projects in their slide decks.
If you have an existing rules-based decision-tree system, and you compare its performance with a bunch of other decision trees, and it does better, you are implementing a random forest that happens to be identical to your original system.
If you talk enough all the models and hyperparameters you compared and suchlike that you experimented with, you can probably sufficiently impress people with the talk about the enormous deep learning model you spent several months developing that they won't even remember you mentioning the two-clause Boolean expression that you actually put into production. And of course it's AI. You used k-fold cross validation to select it.
We did the same thing when I worked for a resume search/sort/share site. Built a big ML tool that could look at job listings and resumes and pick who was best for each job. Our training data set was millions of resumes, hundreds of thousands of jobs, and in most of those jobs, we could say which resumes got shortlisted and which resumes got hired.
In the end, it gave basically the same results as keyword searching. But we marketed the shit out of it.
There is a certain value in understanding why something works and how you can either continously improve it or adjust a few dials when there is an exceptional situation.
Part of the fascination with ML is the (dangerous) myth that you don't have to wrap your head around a complicated problem anymore, instead the solution will just magically fall out on the other side of the blackbox if you just feed it enough data.
Understand ing the intricates of the problems you are dealing with however is a value in itself.
«Part of the fascination with ML is» solving the mystery behind the ability to automatically build functions and behind those functions.
Surely, both in practice and axiologically, understanding and deterministically solving have a great value. Also because of that, the fact that systems exist that can adapt into solutions, but contain a transparency problem ("yes, but why"), contains an immensely fascinating theoretical challenge, in the learning that may come from the attempt to understand the "grown, spawned" (as if a natural phenomenon) system.
The laziness is not necessary: there is a great deal of fascination in unveiling the mysteries in the blackbox.
Then of course, when you have a practical problem to solve (instead of that intellectual challenge and promise), pick your best solution. And surely it is sensible to call it dangerous to rely on something not properly understood, which may hide the potential faults ("yes, we found out it fails here, and it may be that we kind of assumed it "saw" shapes, while really it "sees" textures..."). In professional practice those "active" fascinations (understanding the spawned) may be luxury.
While the methods are very interesting, I often wonder about assumptions about what can be modeled. We already know that it's not possible to correctly predict an arbitrary nonlinear system numerically (since that system could be chaotic).
It's one of the reasons why for specific problems, heuristics or statistics are way better than any attempt at nonlinear modeling / ML prediction (e.g. highly accurate climate models vs. struggling weather models).
I even hear people pitching ML for applications where determinism and explainability aren’t optional, like regulatory and financial reporting for a financial institution.
Indeed! Regulatory reporting is clearly defined, i.e., what needs to be reported and how. Yet I see groups trying to use ML to determine what to report, which makes me think they don't fully understand the topic they are working on.
How do you get promoted and what will you put on your CV if you don't change stuff and just keep the lights on? This is a dilemma all the way from top management to developers. How does a project manager build a career if there are no projects?
Change is needed because people want to have jobs and they will make work for themselves if none exists.
>Change is needed because people want to have jobs and they will make work for themselves if none exists.
Traditional jobs must have solved this problem somehow. You don't usually see e.g. windowmakers or installers coming up with windows in the shapes of superellipses because square windows are already solved, or stoves coming with integrated fridges because "just an oven and a top" is already solved.
Yes, it is called “fashion.” People periodically replace clothing or reconstruct buildings or alter cooking or food presentation. It doesn’t change much but it does maintain a great deal of economic activity, keeps the motors running as it were. By and large operating systems and websites and mature software systems are similar. It is a good thing because it soaks up the attention of people who would delay efforts to evolve, example Microsoft contemplating its OS navel as Netscape came about. It is also good because a small amount of the large economic value of fashion is still more than sufficient for the development of something new. As the “startup” has become fashionable the cycle repeats where the relatively inanimate bones of effort that doesn’t create new value is used as a framework for the rare things that do.
> You don't usually see e.g. windowmakers or installers coming up with windows in the shapes of superellipses because square windows are already solved, or stoves coming with integrated fridges because "just an oven and a top" is already solved.
Well, actually you do. You'd be surprised. Let's ignore for now that there are different window makes and models. There has been a transition from custom-made windows to ready-made windows, whose production cost is higher but the total cost ends up being low because you can have a crew drop by in a construction site and get it done in a few minutes.
Then there's the current progresses in energy efficiency, and also fancy gizmos like actuators and sensors and all kinds of domotics.
Then there's security systems embedded into windows, which further adds sensors and networking and actuators.
You're now far beyond your mom and pop's windows.
And progress ain't done yet. We're starting to see self-dimming windows and also self-cleaning windows.
Most of the window making jobs are just production. In software you don't have the same friction of production. You don't have to type in the source code each time you install the program. But you have to manufacture a new window and physically install it each time which is labor intensive. Also, the window designer job is not hyped as much as IT jobs are.
Furthermore, you do see household appliances getting fitted with useless feature bloat and shoddy software and wireless and touchscreens on microwaves etc. It happens. IoT, subscription based software updates for power drills etc... Tractors that can't be repaired and contain a jumble of proprietary software as a service etc.
A couple of years ago I worked for a bank replacing an in-house library that basically moved and transformed data from one database to another with a highly contrieved Spring Batch solution.
There was absolutely nothing wrong with the "ugly" framework code, it was quite beautiful, well structured, configurable and fast.
Somebody didn't like that you didn't write java code and the properties file based DSL was indeed odd, but nothing wrong with it after you bothered to read the library code.
The Spring Batch code was more explicit, but much uglier, overall.
People have to have list items for their yearly review cycle and their CV. "Replaced a legacy system with a more modern solution" can be presented in a light that earns you cookies. But it may be seen as useless by the higher ups if all they care about is new features. You have to know what impresses your boss and your boss' boss or the interviewer at your next job.
> There was absolutely nothing wrong with the "ugly" framework code, it was quite beautiful, well structured, configurable and fast.
I'm sure there is far more to the story than the new guys doing a poor job writig a replacement service.
For example, old code does add maintenance costs just by the fact that it's either tied to old frameworks or even OSes, either of which might not be maintained anymore. Also, I don't feel it's a honest argument to criticize java for the sake of being java. If there was ever a production-minded programming language and tech stack, that would undoubtedly be java.
I'm sure the guys who designed the replacement service would have a few choice words regarding the old service.
> How do you get promoted and what will you put on your CV if you don't change stuff and just keep the lights on?
Find a new need. Every good product (and many bad products) is an answer to some need. And the world's full of all kinds of needs that we can work on.
However, sometimes we start projects without proving they actually answer a need, or sometimes the internal corporate needs don't match the user's needs (I'm looking at you, integrated advertising in Windows 11).
Finding a new need is risky and difficult. Tweaking and rewriting parts of an existing product with proven market adoption to fit the new fads delivers more predictable flashy results and successes for your CV and career and visibility within the organization.
You have to constantly reinvent yourself, or someone else will and take all your customers.
I'm not saying you are wrong, but you aren't right. There is a balance. You can't stand still, but quality that comes from improving the current thing is important as well.
Presumably what’s meant is it hasn’t improved in 40 years, and even then it was probably just “barely good enough”. This might be considered MVP but that depends on whether you have to actually use it or not.
One thing that I could imagine: it bases it's decision only on few (or the 'wrong') features while you (or marketing) want to consider more.
We have had a project where we were asked if our model would consider X. So we added X to the model but this didn't increase performance. Now the sane, simple answer would be to just ignore X. But then people come and ask why, doubt that it doesn't improve results, competition without ML considers X.
That doesn't happen (or is hidden) in a none ML situation where some decisions aren't questioned by a benchmark.
For very good reason, by an overwhelming majority of developers. The fact that a few developers thought VB.NET was even worse than VB6 doesn't lessen VB6's dreadfulness, it just highlights VB.NET's dreadfulness.
>The final release was version 6 in 1998. On April 8, 2008, Microsoft stopped supporting Visual Basic 6.0 IDE. The Microsoft Visual Basic team still maintains compatibility for Visual Basic 6.0 applications through its "It Just Works" program on supported Windows operating systems.
>In 2014, some software developers still preferred Visual Basic 6.0 over its successor, Visual Basic .NET. Visual Basic 6.0 was selected as the most dreaded programming language by respondents of Stack Overflow's annual developer survey in 2016, 2017, and 2018.
Stack Overflow Developer Survey 2016: Most Dreaded: Visual Basic: 79.5%
I think VB6’s bad reputation is that it is stuck in the 90s. I don’t know if any version of a language from that time would be popular now (let’s leave FORTRAN and COBOL aside).
I disagree that VB.net was dreadful. But it broke backward compatibility but I think for good reasons: arguments being byref by default in VB6, collections being inconsistently 0 based or 1 based, the SET keyword that wasn’t really serving any purpose and was inconsistently applied, having to provide parameters within brackets or between spaces depending on whether the return value is assigned to a variable or not, etc...
I have a lot of sympathy for the frustration of someone who has to maintain a huge code base when backward compatibility is broken, but I think the changes VB.net introduced were necessary.
I remember when my father had to use medicall services billing program supplies by a natinal health insurance company, and he had some problems with it.
Luckily, I was a student in Bucharest and I went to their headquarters to play middleman between my father and their "informatician".
This "informatician" was the sole architect, UX designer, developer, tester, release manager for this program -- VB6+ access.
I sort of helped him debug the code, he built me a special version and handed it to me on a CD.
The program was ok UX wise and blisteringly fast.
Years later, they hired this corrupt company that built software for the State and produced a horrendous program, that took terrible and just the startup took 15 minutes (parsing hunonguous XML and inserting it line by line into a local sql database, as far as I remember reading the logs).
The contract ran into HUNDREDS or millions of euros.
Granted, the scope of the program was a bit wider.
It's not the language itself, but the platform. VB6 was primarily used to write RAD GUI applications. The GUI elements VB6 provide are very outdated by today's standards.
What's wrong with VB.NET Winforms with the Visual Studio WYSIWYG? I still have yet to find a better GUI building experience (alternately the same thing in C#)
It's been unsupported for 12+ years ? If you have code relying on it and haven't migrated to something supported it means your code is not maintained (don't care what your excuse is, using VB6 in 2020 means you're not actively maintaining the project), written 2 decades ago with the coding standards of the era, the original developer team is gone and probably retired and since nobody is actively maintaining it nobody has much knowledge about how it works.
So yeah anything that's still running on VB6 is very likely crap.
> If you have code relying on it and haven't migrated to something supported it means your code is not maintained
No, it does not. It means that Microsoft no longer provides support for the IDE. That does not prevent the developer from maintaining their own VB6 code. With some extra steps, the official IDE and compiler for VB6 can still be installed on Windows 10. Running programs built from VB6 is still supported.
> written 2 decades ago with the coding standards of the era, the original developer team is gone and probably retired
This applies regardless of the programming language to any codebase that has been around for long enough.
>No, it does not. It means that Microsoft no longer provides support for the IDE. That does not prevent the developer from maintaining their own VB6 code. With some extra steps, the official IDE and compiler for VB6 can still be installed on Windows 10. Running programs built from VB6 is still supported.
If you're comfortable with this then I don't think you're actively investing in your software.
>This applies regardless of the programming language to any codebase that has been around for long enough.
No, if you have a team actively maintaining the project you have the knowledge transfer in-house which is the second part of that sentence.
> If you're comfortable with this then I don't think you're actively investing in your software.
What exactly does 'actively investing' mean in this context and why is it needed? If the software is actively maintained so that it continues to meet business requirements, is that not enough?
> No, if you have a team actively maintaining the project you have the knowledge transfer in-house which is the second part of that sentence.
That is orthogonal to what programming language is being used. When the project is actively maintained, knowledge can be transferred regardless of the programming language.
>What exactly does 'actively investing' mean in this context and why is it needed? If the software is actively maintained so that it continues to meet business requirements, is that not enough?
If you're actually investing in maintaining something that's running on a deprecated platform that's decade over EOL and nobody wants to touch with a 10 foot pole - that sounds like a crap project by definition.
Anything that's sufficiently funded to be actively developed would have figured out a migration plan by now, the only scenarios where it wouldn't sound like terrible projects to work on.
>That is orthogonal to what programming language is being used. When the project is actively maintained, knowledge can be transferred regardless of the programming language.
No it's not when the language is deprecated by the owners for over 12 years at this point. It's like having software that only works on windows xp and maintaining it because you can still boot a VM to run it. Good luck working on that POS.
> If you're actually investing in maintaining something that's running on a deprecated platform that's decade over EOL and nobody wants to touch with a 10 foot pole - that sounds like a crap project by definition.
The platform it runs on is Windows 10, which is not deprecated. Microsoft provides an 'It Just Works' guarantee on Windows 10 for VB6 applications. It does not matter whether someone wants to maintain code. The company pays people to do it. Just like how there are many people who do not want to work on proprietary software but do it anyway because their employer pays them to do it.
> Anything that's sufficiently funded to be actively developed would have figured out a migration plan by now, the only scenarios where it wouldn't sound like terrible projects to work on.
Actively developed means that bugs are fixed and features are added as needed by the business. It does not mean jumping on the latest tech trends when there is no business justification. And I am pretty sure that the users are happy that they can use a fast, responsive application instead of a lumbering, bloated Electron app.
> No it's not when the language is deprecated by the owners for over 12 years at this point. It's like having software that only works on windows xp and maintaining it because you can still boot a VM to run it. Good luck working on that POS.
That is a strawman argument because the VB6 IDE and programs compiled with it run on Windows 10 natively, without a VM. And running VB6 programs on Windows 10 is officially supported.
>The platform it runs on is Windows 10, which is not deprecated. Microsoft provides an 'It Just Works' guarantee on Windows 10 for VB6 applications. It does not matter whether someone wants to maintain code. The company pays people to do it. Just like how there are many people who do not want to work on proprietary software but do it anyway because their employer pays them to do it.
My point is that every time I've seen scenarios like this with products stuck on unsupported platforms is that product is used but there's no money in actively maintaining it (or else it would have made migration plans in the last 12 years). This means you are likely getting shit money working on it, working on a legacy stack you won't use anywhere else, codebase is almost always shit, and the work you do is unrewarding. So crap projects by definition, and every testimonial I've seen so far confirms it.
I've worked on projects being stuck on tech close to EOL - they always had migration plans to upgrade to supported tech.
I think none of those assumptions apply in this case, or even in general. If there were no money in it, then the products would not have been actively maintained. VB6 was ranked 20th in TIOBE last year, so it is not just used in my company, either. In fact, if it were so unused, Microsoft would have dropped support for it already, but they currently support it until Windows 10 EOL, and possibly will on Windows 11 as well. I have not looked at the codebase behind the VB6 products in question, but there is no reason for me to assume that it looks any different to any other long-established codebase, or that the developers who maintain it are paid less. I would assume they are paid more because they are harder to replace, which in turn makes their work more rewarding.
I have worked on products with old tech stacks as well, e.g. a C++98 codebase for the core product of a multi-billion-dollar company, with no plans to migrate. New features were being frequently added, and the company's own standard library replacement that bridged the gap was itself actively developed.
At my last job there was a team of 4 or so people who had originally written some vb6 code that they were still maintaining. This was as recent as 2020 and since there were no plans to stop I assume it's still ongoing with, at best, some plans to move off being made what with how slow things moved.
Welp, what if I told you a Sixth Form (Y12, age 16-17) in a UK school has a computing course that just started and is reportedly using VB6 ... I'm really not sure what to say?
The "the right tool for the right job" applies for ML topics too.
If the job involves "looking smart and innovative" for whatever reasons, people tend to err on the side of overly complex solutions.
On the other hand if the advice "let's just go with an SQL query built on a stack of egregiously oversimplifying assumptions" comes from someone, who doesn't know how SQL and linear regression / logistic regression with binning/bucketing / simple decision trees work, I would ask for a second opinion. Because a huge part of the retail banking, non-life insurance and marketing business is running on this simple stack. Obviously profitable.
If the same advice comes from someone, who knows when to use deep learning instead of XGBoost and why, I would go with his/her advice. And I would try to keep him happy and on my team.
My workplace has got all kinds of attention for building a blockchain based data collection system that encompasses an entire sector of the economy. It's "almost done", so we are right now starting a simple set of REST services that write into a badly normalized transactional database just in case it stays "almost done" for too long.
"When you have a problem, build two solutions - a deep Bayesian transformer running on multicloud Kubernetes and a SQL query built on a stack of egregiously oversimplifying assumptions. Put one on your resume, the other in production. Everyone goes home happy."