Took out the sender unit and little angled piece. I didn't notice any fresh oil leaking but I put some ultra black on both piece's threads. Put everything back in.
You put silicone on the threads? Not only is that not the appropriate sealant for threads, the sending unit and corresponding hole are pipe threads (aka 'NPT') and don't require sealant.
It was designed that way, because there needs to be good metal-to-metal contact between the electric sending unit and the engine block for it to work right. Sealants prevent that metal-to-metal contact.
Now you get to clean that silicone out of that hard to get to hole, when you eventually replace your sending unit.