Is this what I need? We have ad hoc business processes like:
1. Sell service to client
2. Email client with scheduling info
3. Agree to scheduling date with client
4. Send client email with documentation and request their data
5. Remind client to send data
6. Alert admins data hasn't been received
7. Beat data out of client
8. Prepare data
9. QA and checkoff service
10. Email client to review and approve
11. Publish/run service
12. Send client stats/results
And I want to move that from spreadsheets, personal emails and admins keeping in their head where everything is at, to web forms and uploads, automated emails and dashboards. I looked at airtable, smartsheet, budibase and many others but they seem to be very project-based, where you are inventing a unique workflow for each project and using it to keep managers on top of where their team is at with the various steps. Project Management vs process management. None of them seem to have decent calendar integration. Few of them seem to be great about emails or scheduled scripts.
I have APIs for my data, or I can code them if needed. I'd prefer a low-code to a no-code approach, where managers have a spreadsheet view and can do some of of the UI work and programmers can make it do stuff and handle integrations.
Yes, windmill would likely be a great fit for that as long as you're willing to do a little bit of coding.
For instance, we do not have a calendar integration per se but can manage your oauth tokens towards any service (including gcalendar) and allow you to write code using their official sdks.
For spreadsheets, you can build a similar interface in windmill using aggrid.
It seems you also need approval steps and have your workflows wait for events, and that's built-in in our workflows.
If you want to explore this I'd recommend taking the approach I've seen with scripts that's very successful -
The first bash script simply tells a person what to do and hit enter when they've done it.
Then the easiest to automate parts are automated, as and when it's valuable to do so.
Windmill appears to have manual approval steps - so you could try modelling things just with those. Then automate the most annoying/costly/easy to automate steps one by one.
This should be easier to create, and actually solves an initial problem of knowing where everything is up to. If it doesn't help, or the process is too inflexible when reality hits it, you won't have spent too long automating things.
I don't think that'll achieve your goals, it's far too clunky and inflexible for actually running a business end-to-end. (No knock on Windmill, they're all like this in my experience as the way they model the world is how a programmer thinks about things, not a business owner. BPMN is straight up cancer—avoid.)
I would suggest looking at workflow systems within the VFX space, which they call "pipelines". They are human-driven and maintained, but support very high levels of process automation.
VFX pipelines are written in Python, but the key thing is, they're very good at a few things simultaneously:
(1) Highly irregular work…
(2) That nevertheless requires lots of process, including of the ad hoc variety…
(3) With high human touch (human in the loop), so calendars, task assignments, timelines, approvals, producer dashboards, etc.
(4) That's repeatable on the next project/opportunity, with as much customization as needed.
As a bonus, pipelines handle billions of dollars of work annually under high pressure, with tens of thousands of users, so there's a lot of hard won experience built into them.
I personally have found that there are three levels of VFX pipeline tech that need to be combined and customized to do everything. This is the most recent stack I've set up:
* Kitsu for the producer/manager level, human task assignments. Mainly for managers, has the calendar functionality.
* Prism Pipeline for software integration (i.e. keeping the humans following the process). This is what people actually do.
* Kabaret for compute work. People usually kick off these jobs, but they're handled by the farm. You'll be responsible for everything that happens here. [0]
Don't try to get by with just one or two of the three, even though feature-wise it looks like you might be able to. Incorporating computer-driven workflows is tricky and requires purpose-built libraries in my experience. Just use the right library for the job, and accept that they all have somewhat overlapping functionality.
For most non-VFX work, you'll need to script access to the browser, since that's where people do stuff today (i.e. with the Prism Pipeline library). Launch Chrome, Firefox, or Edge with a debug port and use Playwright as your "plugin" mechanism, with Prism/Python as the driver. You can make your pages do whatever you want and script literally any web-based process, filling in fields from your database as needed, adding data to the page, etc.
For the last six years, I've run a business that does around $5M of annual revenue using this approach, with 4 FTE people (including me). We handle around 40 million compute tasks a day, but also have all of the "daily work" where humans are in the loop. We do all of the tasks of the type you've listed.
HTH!
[0] If you need fully automated task generation (so, no human in the loop), write "pump" scripts in plain Python that push jobs into Kitsu or Kabaret at a particular interval. I personally run them under a god monitor, and have them exit after, say, 15 minutes (or whatever interval makes sense) to get cron-like functionality. This is useful because you can pause execution of any pump process easily, change the interval, restart them, etc. plus you get easy access to the logs for debugging. The scripts themselves are usually less than 100 LOC. I have around 60 of these that run the automatic parts of our business.
Whenever I need custom, temporary behavior in this part of the business, I copy the pump script, edit the original to avoid the special case, and have the new script only handle the special case (with whatever new behavior I need). Done, takes very little time, and is easy to run temporarily (e.g. for the next week, a common occurrence because some downstream vendor has caused a problem we need to work around).
Another approach to trigger tasks/workflows is to have a server that gets pinged by 3rd parties (e.g. Webhooks) and does the same whenever the endpoint is invoked. We have that too, but it's literally just a few scripts. (Most workflow systems seem to believe this is damn near 100% of the stuff that needs to be "automated.")
Searching for "Vfx pipeline" just gave me visual effects stuff. Quite sure that was not what you talked about. Could you explain a bit more what ypu mean by it and point to a resource. Keen to learn more.
I think windmill could have helped with several of the task and automated some. Its very easy to create dashboard and table with statuses. But its def not a process tool, if that was what he was after.
All of these projects are targeted at VFX/animation, but there's nothing about them that ties them to those things, it's just why they were developed. You can use them to run any kind of business IMO.
I posted about it because (a) they handle the commenter's use case, and (b) most HN people are probably unaware they exist and would be discouraged into thinking they are only useful for VFX/animation when in fact they are general approaches to running a business that requires high interaction between humans, processes, and background jobs in a very dynamic environment.
Jobber looks like it is more in the "project management" space. All of my processes are post-sales. We're not inventing new projects that we need to organize. We have a short list of services we sell and each of them has a process with a lot of dependent steps that require a lot of communication and coordination.
> We have a short list of services we sell and each of them has a process with a lot of dependent steps that require a lot of communication and coordination.
Something that comes to mind: In Jira my administrators have locked it down so I can't move certain tickets to certain statuses
1. Sell service to client
2. Email client with scheduling info
3. Agree to scheduling date with client
4. Send client email with documentation and request their data
5. Remind client to send data
6. Alert admins data hasn't been received
7. Beat data out of client
8. Prepare data
9. QA and checkoff service
10. Email client to review and approve
11. Publish/run service
12. Send client stats/results
And I want to move that from spreadsheets, personal emails and admins keeping in their head where everything is at, to web forms and uploads, automated emails and dashboards. I looked at airtable, smartsheet, budibase and many others but they seem to be very project-based, where you are inventing a unique workflow for each project and using it to keep managers on top of where their team is at with the various steps. Project Management vs process management. None of them seem to have decent calendar integration. Few of them seem to be great about emails or scheduled scripts.
I have APIs for my data, or I can code them if needed. I'd prefer a low-code to a no-code approach, where managers have a spreadsheet view and can do some of of the UI work and programmers can make it do stuff and handle integrations.