This has been similar to my experience. Rules Engines like CLIPS are just outside of the bounds of "cheap enough to spend time learning on the off-chance it is indeed better than what we've already written."
Not to self-promote too much, but the main focus of my programming efforts is currently CLIPS. Over the past few years, I've written articles about and even made some fun projects with CLIPS:
Is it possible to make a reasonable rules engine in Ruby rather than a "bolt-on" (separate rules language files)? I think its interesting that its the LISP languages that spawned rules-engines, and as far as I've seen, the only language that can produce a reasonable rules engine as an API library.
The precursor to CLIPS was called ART Inference and was a large commercial & very expensive Expert System development tool, written in Common Lisp. Unfortunately it seems to be lost - I haven't seen anything about it for several decades.
Other than that there are a bunch of rule-based systems in Common Lisp.
For example LispWorks Enterprise includes "KnowledgeWorks", which features a forward chainer based on the RETE algorithm.
This is the second part in a series for writing Ruby extensions in C. This article focuses on converting Ruby VALUEs into CLIPSValues so that you can assert values from a Ruby program into a CLIPS Rules Engine.
Incredible. The usual response to "should I host my own email" is "don't do it; you'll get hacked."
Three questions:
1. Have you heard of this complaint?
2. Do you use a home ISP connection, or a commercial ISP connection? A "home ISP connection" here usually comes with a dynamic IP address; you can't get your hands on a static address without paying a very large amount monthly or getting a commercial connection.
3. You say "I don't expose anything to the public internet unless absolutely necessary." Is your ip address via your domain name one of those "necessary" items?
1. Yes, most people will tell you not to host your own email, because its too complicated/difficult to get your mail delivered reliably.
A lot of this is FUD. Yes, email is a bit more difficult to get right than say, hosting a web app behind Nginx. It's an old protocol, with many "features" bolted on years later to combat spam.
I'm not sure how email is easier to "hack," unless there is a zero day in Postfix or something. Back in the day, lots of script kiddies would find poorly configured mail servers that were happy to act as an open relay...maybe the stigma persists?
To deliver mail reliably, you need 4 things (in my experience):
- A static, public IP address with a good reputation (ie, not on any spam blacklists)
- A reverse DNS record that resolves back to your mail server's IP
- A domain SPF record that says that your mail server is allowed to deliver mail
- DKIM records and proper signing of outgoing messages (DMARC records help too)
2. I have a residential cable internet connection, but pay extra for static IPs. You can probably get by with a dynamic IP and some kind of dynamic DNS service, as long as you don't want to send email. You could still receive email locally if your MX recorded pointed to some kind of dynamic DNS record.
Note that some ISPs explicitly block outbound traffic on port 25 due to spammers. You might need to check with yours.
3. The only things I expose to the internet are Postfix (to send/receive emails), XMPP (to chat with others), and my web server. Everything else (calendar/contacts, IMAP, Syncthing, etc) stays behind my firewall, accessible only to internal hosts. I use wireguard on my Android phone to access these services seamlessly when I leave the house.
I've never bothered to conceal my IP address. For awhile, I experimented with using Mullvad VPN for all my egress traffic. Unfortunately I spent all day solving CAPTCHAs...wasn't worth it (for me, anyway).
EDIT: I should add, that I also have a "normie" email address at one of the usual providers that I use for really important things like bank accounts / utility providers. If I get hit by a bus, I don't want my (very nontechnical) wife to deal with sysadminning on top of my early death.
For all our personal communications though, we use my selfhosted email domain.
It's not that hard to do. Harder for residential address blocks for sure. But if you do all the other things previously mentioned like SPF/DKIM etc then cleaning up an IP address isn't that hard.
The only service we've ever had issues with is Outlook as they'll ban whole block for opaque reasons and we just escalate it to the provider and they sort it. We just moved two self-hosted mail servers to new IP addresses and there were only 2 lists to clear them from, which was a fill in form style automated process to resolve.
There's always SES (or other service of choice) as a backup for sending anyway if you notice something getting blocked. It's easy to switch to that for a day or two whilst you resolve an issue - though I must admit I think we only had to do that once in the last 12 months.
Maybe I'm breaking some kind of sysadmin code here and I don't realise it's a secret that self-hosting email isn't that hard? Am I supposed to keep up the myth that it is? :-) Any greybeards here please let me know!
I played around a bit with sending via SES and Sendgrid. I generally found that deliverability on either of those was actually worse than even one of my slightly dirty IPs.
Previously, I was also using Sendgrid as well. But they seemed to start doing the "growth at any costs" bullshit which for an email sending company means accepting and delivering spam. (Regardless of their PR/weasel-words these places use to deny it, that's what it comes down to). Thus lots of places now just drop all mail that comes from Sendgrid, no workaround.
When that happened, a friend pointed me to smtp2go, which I've used since personally and we now use at work. We haven't (yet) had anything blocked as spam (less than 10k emails sent a month though), so it seems like they've not done the "growth at any costs" bullshit like Sendgrid.
You're not the first person I've heard say that. It's interesting that we haven't faced that issue. I wonder if we'll get a nasty surprise the next time we try as it has been a while since the last time we did it.
Also note that it's super easy to configure postfix (and likely others) to send all outbound email via a third party service.
I personally use smtp2go.com, and was on their free tier for ages (now upgraded via work). Can recommend, as it "just works" and avoids all the mucking around with SPF/DKIM/etc.
Oh, on a similar note, definitely avoid Sendgrid if you want to send email via a third party. They're outright blocked (as a spam source) by way too many places to be considered reliable any more. :(
DKIM and DMARC are not anti-spam techniques per se. They are used to verify that the message is authentic, and that sender is authorized to send email on behalf of the domain.
If the sender is passing as an authorized sender (DMARC aligned), then the receiver has a pretty good indication the email is legit and that the sender was delegated to sent email on behalf of the domain. If the email is then classified as spam (based on its contents), then it is easier for the receiver to choose whether to adjust the reputation of the domain (in case of DMARC alignment), or the IP (if not aligned).
A DKIM signature and DMARC alignment is no guarantee that the email passes spam filters. The whole point of DMARC is to give the receiver as much information as possible to make a confident decision on the legitimacy of the email, and the reputability of a domain.
DMARC and DKIM works both ways, if you are sending legit email (not spam), it will improve your deliverability, but if you are in fact spamming then DMARC will reduce your deliverability (as it should).
I've done it for a couple of years, all traffic comes into the VPS and Wireguard immediately redirects to my home machine VM. I can take the VM down, bring it up on another machine, it calls to my VPS to the Wireguard server, establishes the tunnel and then my email and web are now going to the VM on the new home machine, or whereever in the world I want to bring that VM up. Yet, to any clients hitting my public IP (the cloud VPS), nothing has changed except for a few minutes downtime.
> 2. Do you use a home ISP connection, or a commercial ISP connection? A "home ISP connection" here usually comes with a dynamic IP address; you can't get your hands on a static address without paying a very large amount monthly or getting a commercial connection.
Weirdly, most of the ISP's I've had on the NBN here in Australia were happy to give me a static IPv4 address for free (and my current one will set you up an IPv6 /56 block, but its beta apparently).
I just finished writing this article describing how to write a multi-user environment in CLIPS and Go. This article describes how one might write an arbitrary web application that users can connect to and use over TCP/IP.
I hope that it can help some folks out there get the gears turning on what is possible with CLIPS! I'd love to hear feedback on what you all think.
Thanks! Really good feedback. I'll be adding on to this overtime, and this would be a great addition.
Games are the easiest analogy because they describe their features as "rules," but I've seen Rules Engines/Expert Systems reveal themselves organically across multiple domains. Try applying the concept of "Rules" that trigger when data in your system matches a certain pattern.
