For a long time now, I've been kinda self-experimenting on inducing these kinds of states in myself, without resorting to hardware or hard/scary drugs.
It all started with my realization that sleep brought me answers in general; i.e. a realization that my general pattern was to work on hard problems all day until I hit various brick walls, then step away and do Real Life and then later at night I'd Sleep, and then often if I'm lucky I wake up the next morning with important realizations that get me past the previous day's brick walls, and then the cycle repeats.
Once I realized this pattern, I started optimizing for more sleep cycles per day and a more-immediate transition from BrickWalls->Sleep, basically by injecting a daytime nap in the midst of my workday if/when I can (working from home has its privileges!), aiming for ~1.5h or ~3h nap increments when I can, since that seems to match well with my sleep cycles for reaching REM in the middle and then waking back out of it on a natural cycle boundary.
I'm also a heavy coffee drinker for the stimulant effects on my thinking, and at one point heard about the "Coffee Nap" idea (TL;DR - it takes ~30m for caffeine to really kick in, so when you get tired you chug a coffee and lay down, and let yourself wake later as it kicks in).
Somehow over time all my random experimentation on these various inter-related things settled into a new pattern that works well when I can achieve it: I try to hit the brick wall while still fairly amped on caffeine (have a last cup as I'm moving away from the laptop if I'm behind on my caffeine intake), then switch over to a nap state.
As I'm laying in bed falling asleep, I initially force my thoughts on whatever my Brick Walls are, and my body's still a bit uncomfortably caffeine amped/buzzed for laying down and trying to sleep, but between the coffee buzzing and the descent into sleep, I now usually end up having a very odd transitional but semi-conscious mental state for a solid 10-15 minutes (sometimes longer) on my way to sleep. During this time, free associations and strange dreamy unrelated things start springboarding out from my initial more-directed thoughts, and it's clear this is basically an open channel to get those lateral-thinking associations going while still barely conscious. It's like a psychedelic-drug-free version of a short acid trip or useful lucid dreaming state. Eventually I succumb to sleep, but I have had a very good success rate (relatively - it's still not necessarily high in the absolute!) at getting good intuitive answers to my problems on waking.
Over time I've gotten better and better at achieving these states with practice, but it still requires a lot of these triggering conditions/patterns above. I think/hope eventually I may reach a point where I can induce these mental states at will without actually going to sleep afterwards or relying on caffeine buzz to kick it off.
I recently read "A mind for numbers" which leans on this concept of working hard on a problem (focus mode) and then stepping away from it for some time (diffuse mode) very heavily. I also realized that this works really well for me and i guess for most people.
I've relied on this for decades for complex development tasks. Now when a manager asks me to propose some design solution, I just say flat out that I'll probably know what to do after a couple of sleeps.
It all started with my realization that sleep brought me answers in general; i.e. a realization that my general pattern was to work on hard problems all day until I hit various brick walls, then step away and do Real Life and then later at night I'd Sleep, and then often if I'm lucky I wake up the next morning with important realizations that get me past the previous day's brick walls, and then the cycle repeats.
Once I realized this pattern, I started optimizing for more sleep cycles per day and a more-immediate transition from BrickWalls->Sleep, basically by injecting a daytime nap in the midst of my workday if/when I can (working from home has its privileges!), aiming for ~1.5h or ~3h nap increments when I can, since that seems to match well with my sleep cycles for reaching REM in the middle and then waking back out of it on a natural cycle boundary.
I'm also a heavy coffee drinker for the stimulant effects on my thinking, and at one point heard about the "Coffee Nap" idea (TL;DR - it takes ~30m for caffeine to really kick in, so when you get tired you chug a coffee and lay down, and let yourself wake later as it kicks in).
Somehow over time all my random experimentation on these various inter-related things settled into a new pattern that works well when I can achieve it: I try to hit the brick wall while still fairly amped on caffeine (have a last cup as I'm moving away from the laptop if I'm behind on my caffeine intake), then switch over to a nap state.
As I'm laying in bed falling asleep, I initially force my thoughts on whatever my Brick Walls are, and my body's still a bit uncomfortably caffeine amped/buzzed for laying down and trying to sleep, but between the coffee buzzing and the descent into sleep, I now usually end up having a very odd transitional but semi-conscious mental state for a solid 10-15 minutes (sometimes longer) on my way to sleep. During this time, free associations and strange dreamy unrelated things start springboarding out from my initial more-directed thoughts, and it's clear this is basically an open channel to get those lateral-thinking associations going while still barely conscious. It's like a psychedelic-drug-free version of a short acid trip or useful lucid dreaming state. Eventually I succumb to sleep, but I have had a very good success rate (relatively - it's still not necessarily high in the absolute!) at getting good intuitive answers to my problems on waking.
Over time I've gotten better and better at achieving these states with practice, but it still requires a lot of these triggering conditions/patterns above. I think/hope eventually I may reach a point where I can induce these mental states at will without actually going to sleep afterwards or relying on caffeine buzz to kick it off.