You just described pretty much every day of my life for the past 8 years.
I think there's nothing wrong with this. (Though of course I'm biased.) The intermittent fasting seems to benefit my concentration and energy, even if the fasting happens to be unintentional and due to distraction or laziness. The only externality is if you end up eating a lot right before sleeping, which tends to impact my sleep and digestion. As long as you avoid that and are getting enough calories and nutrition, I don't see the issue.
1. Fast all day out of laziness: no food in the house, in the middle of something, one more thing, etc. etc.
2. Get incredibly hungry all of the sudden
3. Optionally push through for a couple more hours upon which my hunger subsides somewhat
4. Desperately need food, upon which I order out or walk down the street for something
I end up eating 1-2 meals a day, and while many tell me I'm basically doing an awesome job at intermittent fasting, it doesn't feel that way.
I'm trying to cook more.