Accidentally stealing something where you don't realize that you stole it, and the store doesn't realize that you stole it, costs less than what happened. In terms of net global utility, even taking into account that your sense of morality has utility to you, it costs less. So this isn't a good example of "good behavior being rewarded" (i.e. having a net-positive outcome in global utility.)
To be clear, theft isn't theft (legally or ethically) without mens rea — if you accidentally walk out of a store with something, you haven't done anything wrong.
And if you later realize you now have possession of the thing, that doesn't mean you're suddenly guilty of theft, either. Theft is an act requiring mens rea, not a state of someone who possesses something that was at some point illegitimately dispossessed from its owner. (If the latter was true, then most people who buy anything from pawn shops would be guilty of theft, because those things were often stolen from someone at some point.)
You might, separately, see there being positive ethical value in returning an item to its owner; but this ethical value should be considered independently from the ethical cost of having stolen (or not stolen) the thing. If you think there's positive ethical value in returning things to their owners, then you'd likely feel an ethical impoetus to e.g. "steal" stolen bicycles from those piles you see on the front yards of flop houses, and bring them back to the people who own them. (Even though those people have probably bought new bicycles already, and won't know what to do with the returned ones. Even though the stolen pile of bicycles is likely just a pile of rust due to lack of maintenance.)
You're falsely generalizing your specific situation into a full proof for a general ethical policy, though.
What if, instead of the same day, it had been the next day when you noticed you now had the drill bit, long after you brought it home? Would you still think you had an imperative to return it? Would it still feel like deliberate theft?
How about if you didn't notice that you had it, and it fell out of your pocket at some point on your way home, and then you realized the next day that that had happened? Would you feel an ethical wrongness that could only be righted by retracing your steps to find the store's lost drill bit, so it could be returned? Or would you accept this as just a series of accidents with neutral ethical value — like e.g. goods being damaged in shipping?
What if, instead of a drill bit, you found after you walked out of the store that you had a $5 bill stuck to your shoe? Would you feel the need to walk back inside and ask who lost the $5 bill?
To be clear, theft isn't theft (legally or ethically) without mens rea — if you accidentally walk out of a store with something, you haven't done anything wrong.
And if you later realize you now have possession of the thing, that doesn't mean you're suddenly guilty of theft, either. Theft is an act requiring mens rea, not a state of someone who possesses something that was at some point illegitimately dispossessed from its owner. (If the latter was true, then most people who buy anything from pawn shops would be guilty of theft, because those things were often stolen from someone at some point.)
You might, separately, see there being positive ethical value in returning an item to its owner; but this ethical value should be considered independently from the ethical cost of having stolen (or not stolen) the thing. If you think there's positive ethical value in returning things to their owners, then you'd likely feel an ethical impoetus to e.g. "steal" stolen bicycles from those piles you see on the front yards of flop houses, and bring them back to the people who own them. (Even though those people have probably bought new bicycles already, and won't know what to do with the returned ones. Even though the stolen pile of bicycles is likely just a pile of rust due to lack of maintenance.)