Push only when K9 is open?

2

You can disable the constant notification. See How to configure Push?