There is a sort of standard sequence of finishing fine wood - stain, filler, (varnish or oil).
The filler is a heavy, goopy kind of stuff that you cut with turps to about the consisty of heavy cream. You can get this kind of filler from places like Woodcrafters or Constantine or any other purveyor of fine furniture finishing materials.
You brush it into the grain, let it partially dry and then rub it with coarse burlap AGAINST the grain. What remains is the stuff that has been forced into the pores/grain.
This stuff is STANDARD in the wood finishing trade. Get the best you can - the stuff found in local hardware or paint shops is most likely crap.
GIVE IT A COUPLE OF DAYS TO DRY/HARDEN. Most amateurs rush, rush, rush. Put the stuff on, wipe it off and go on holiday. Then, any finish you apply will not lift the filler, will go on nicely, smoothly and IF YOU LET THE DAMN FINISH HARDEN FOR A FEW DAYS will solve your problems. You can then apply as many coats as you wish, giving each a few days AT LEAST, to HARDEN (not only dry, but harden)
Stay away from local hardware store stuff and get specialty woodworker finish stuff.
You CANNOT "kill 2 birds with one stone" in wood finishing. You have to do it one step at a time. I cannot emphasize this enough. I've been making (as a hobby) fine furniture for the past 35 yrs and have learned this over and over. There is no reason why a coat of finish should lift filler unless you didn't let the filler harden. Give it a day or two. The only filler that should remain is that in the pores - nothing should be on the high parts of the wood. Once you're at this point, go away for a few days. Then come back and put a VERY thin coat of finish on it and go away again for a few days.
Keep diong this.
In a month or so you will have a stock that will run up laundry bills amongst those who see it