It is always better to insulate on the outside of basements and crawl spaces, but usually more expensive and more trouble.
Brick or field stone foundations must not be insulated on the inside for much more than one foot below ground level. The poor quality of the mortar in these foundations deteriorates when cut off from the heat of the house -- not the mention the problems of ad-freezing. Interior insulation may be applied to these walls but only from the floor above to one foot below grade, no lower on the inside. Insulation can be applied full height to the outside, as it will actually protect the field stones or bricks from the outside weather. Insulation on either side will do little good if there is an air space between the insulation and the wall. On the interior, we parge the wall flat first. On the outside, pour sand between the insulation and the rough wall as you slowly back fill on the other side -- preventing the soil form breaking up the foam on the rough surface. The walls could easily be covered with sprayed on polyurethane foam -- but this material would probably absorb a lot of water while allowing the wall behind to freeze -- not a good idea. Sprayed on polyurethane if not recommended for below grade use.
How to insulate concrete block walls is a hot question, thanks to their air channels and their mortar joints. Technically speaking, the row of blocks at grade level is supposed to be full blocks, not hollow -- or at least filled with mortar, to prevent air movement from the top to the bottom and back again. To allow these blocks to fill up with water (because you have waterproofed the inside) and then freeze (because you have insulated on the inside) is just asking for the blocks to completely break into vertical cracks from ice block freezing. The deciding factor on insulating may be whether the drainage on the outside is adequate to avoid ad-freezing problems in the joints. Insulating on the outside of concrete blocks brings all the weaknesses of these walls into a warm protective shield.
Poured-in-place concrete walls appear to survive interior insulation with the fewest problems.
Among the factors that may prevent you from taking the preferred route of insulating from the outside are: structural obstructions (steps, porches, driveways); legal obstructions (property line encroachment, historic building restrictions); aesthetic obstructions (the "look" of the house or row of houses, untouchable plants); and financial obstructions (the cost of digging and re-landscaping).
More information to fuel the fires of controversy:
-- Combining inside and outside insulation can be done when necessary to go around obstacles. Overlap by three or four feet to prevent thermal bridging through the wall.
-- Exterior insulation only 600 mm (2 ft.) below ground level is generally considered a viable way to insulate. A trench can be dug by shovel with minimum damage to landscaping. Hard-core conservationists, however, argue that this allows an unreasonable loss of energy and would have you insulate deeper. You can save more energy by insulating two feet down and then two feet out horizontally (with a slight drainage slope away from the house). This is easy because when you dig down, you usually dig out two feet anyway. This is almost as effective as 4 feet down if you don't have a water saturated soil that has unusually good thermal conductivity. It you can go up above the floor level (as shown in the drawing) on the outside you can get really effective results because you cover the difficult to insulate header/floor joist area and overlap with the insulation in the wall above.
-- Interior insulation is often chosen because it can be done slowly and the work extended into the winter months. The project usually provides a finished wall at the same time for general upgrading of the basement. The same interior decorating effects can be accomplished directly on exterior insulated walls with cement coating materials (they even come in colors) or by gluing prefinished panels directly to the walls or to strapping. This has the added advantage of leaving the floor space inside undiminished.
-- Both interior and exterior options require protective coverings for the insulation. Inside: vapour barriers and fire protection; Outside: mechanical and sun protection above grade.
-- Exterior insulation requires protection from damage above ground and a flashing over the top that fits up under the house siding air is caulked to the masonry or stucco. This protection can be any of various cement-type coatings troweled on over a galvanized or fiberglass mesh. (Cracks will set in without the wire mesh.) Sheet materials embedded in the ground and flanged at the top will also offer protection. Use cement millboard or 12mm (1/2 in.) pressure-treated plywood.
-- Interior insulation requires fire protection if it's a foam board, and protection of the air/vapour barrier regardless of what material is used. 12 mm (1/2 in.) drywall or 6 mm (1/4 in.) masonite-type prefinished panels satisfy both these requirements.
-- Higher insulation levels can often be obtained by insulating both inside and outside the basement. The wall sandwiched in the middle remains much warmer than if there was no insulation on the outside at all.
-- Deteriorating walls can often be saved by adding exterior insulation, thus preventing the temperature and water from affecting the walls further.
-- Check out Air Spaces in Walls - Myth and Science.