Initial and running costs are essential to consider because they determine how much you save in the long run. Electricity furnaces are the cheapest to buy and install. With gas and oil furnaces, you have to buy other parts like exhausts and make a new connection to the power grid.