It is frustrating when walkway or driveway paving stones heave with frost, or drop with erosion. If properly built, this doesn't happen. Actually what I have written here about paving stones applies to asphalt driveways as well.
You need to layer and compact all the layers below the stones themselves. Inadequate foundation preparation will always cause problems on the surface. The first step is to excavate deep enough. You need to decide where you want the top of the stones, taking into account the slope you will need for water to run off and away from the house foundation. Then you need to dig deep enough to account for the thickness of your particular paving stones, and about an inch of screening sand just under the stones, and with the all important road gravel layer below that. For a walkway, the road gravel should be 4 to 6 inches deep, for a regular driveway, 6 to 12 inches deep. If you are parking a truck or an RV on this driveway, the road gravel layer should be 18 inches thick.
Road Gravel is not just any gravel. It is a mixture of very fine sand, not so fine sand, small rocks and rocks up to 3/4 inch in size -- sometimes called 0-3/4. This mixture of sizes does something very different than the 3/4 inch clean gravel that we use for drainage around the foundation of the house. Because of all the different sizes, it is possible to compact this gravel very solidly. In fact, this is what is used under our roads. You need to compact the construction of a walkway or driveway at every step of the way. In the second photo, you see a piece of plywood on the end of a 4x4 as a tamping tool for small patches. For a full sized walkway you would definitely need to rent a gas powered compacter. You fill a couple inches of material and then compact. If you fill too much and try to compact too much at a time, it doesn't get compacted on the bottom and that will eventually cause sinkage -- a common problem with fly-by-night contractors in a rush. This compacted 'bed' is what carries the weight of whatever is going to be on top.
Before putting down an inch or two of screening sand, many people like to put down some kind of filter cloth that will prevent the screening sand from filtering down and eroding the support under the paving stones. Screening sand is primarily a fine sand with some rocks. Whereas, road gravel is composed mostly of rocks with enough fine sand to fill in between the rocks. The purpose of the screening layer is to allow the precise levelling or sloping of the surface under the paving stones. This layer, too, needs to be compacted. It needs to be just thick enough to get the precise surface you want.
A common error in laying paving stones is to lay the stones without a retaining guard on the outside edge. Without this retaining guard, such as the one in the third photo above, the stones on the edge eventually tip outward. Note that preparation of the sub-surface should be several inches wider than the finished paving stones. This gives good support to the outside stones, as well as to their retaining guard.
After the stones are placed, a very fine fill sand is swept into all the cracks and a final pass with the compactor helps to settle that sand tightly between all the stones.
We tend to concentrate on laying the pattern of stones. Instead, we should really be concentrating on building a properly compacted substrate that is deep enough to carry the load that will be on top. This way, your carefully laid pattern will remain as beautiful as when it was first laid.
When you get some oil on those stones, you really should try the rather remarkable Oil Lift.