It is actually incredibly easy! If you are using a voip line, it is just a configurable field in the UI. You can do it with any voip phone app (e.g. [1]) and a voip provider (e.g. [2]). I have an old archived video showing it here [3]. It is not so interesting though, just me poking around in a voip provider's UI.
To address the other question about phone providers verifying stuff. SHAKEN/STIR [4] protocols are supposed to address this, but I think the telcos are still in ramp up time.
It's ridiculous that phone companies allow this. Anyone wanting to set caller ID via voip should be forced to provide some sort of verification that the number is theirs and the phone company should not route it if it fails verification.
We only have 3 major cell carriers here is Switzerland, it should be trivial for the 3 to verify each other's numbers to see if those customers even exist. Unlike the US each cell provider has his own number prefix. Numbers are portable but only between certain providers.
The iOS feature for silence unknown callers has really been a blessing to my life. I have screenshots of me getting ~10 blocked calls per day. I turn it off when I'm expecting a call, but it saves me so much distraction and rage about my vehicle's extended warranty.
It's not as easy... For example, it is possible and legal to use your own number to call from a VoIP provider, so the recipient can call you back on your actual phone.
On the other hand, it should be possible to detect at least a percentage of spoofed caller IDs and block them (e.g. non-existing numbers).
The VoIP provider could forward the call to your phone as a middleman, or there could be 3 numbers(1. Who to bill 2. Calling number 3. Reply-to number) and only #3 is user-configurable.
The idea of using a VoIP provider would be to _avoid_ using one's phone, e.g. to get better rates... And the reply-to number, it's unfortunately not the way telcos work... and changing all the infrastructure for this would be very very expensive.
https://tacix.at/experiments/signal