Sounds likely that its the foot valve at the bottom of your well. Could be its got something jammed in it preventing it from sealing so it leaks back down once the pump shut off. It might dislodge itself or it might just be warn out and needing replacement. It's not the nicest time of year to be pulling a well line if you know what I mean but as long as you have a pitless adapter on your well casing. In order to pull it up you need to make up a puller using a 6 foot piece of galvanized pipe and a T so it has a handle to turn with. You can get one made up at most hardware stores pretty cheap, just ask em they know what you need. Paint it some bright color before you put it away and stash it someplace. Whatever you do don't tell anyone you have it or it will belong to some other guy the next time you need the thing.
The end has a male thread which will screw into the pitless adapter's top which then allows you to lift up on it and the whole thing can be grabbed and pulled up by the poly rope. Have some gloves as its slimy and a second set of hands is very helpful too. It wouldn't hurt to change the o ring on the pitless adapter for good measure before setting it back in place too. Of course before you get into doing this make real sure its not a leak somewhere easier to get at like the worm clamps before on your upper connections before you get into pulling the pump. If you have to pull it you might consider just shutting it off at the breaker at night for the time being and waiting for some nice warmer weather. Pullin wells really sux in the winter
