Houses appreciate too. The materials and labor cost required to build my house have outpaced inflation by far.

This is most clear in insurance data where replacement cost is isolated from land value.

Houses depreciate if there's an adequate supply of newer housing keeping up with the housing demand of the area. If there isn't then the general GDP growth of the area in which the house is located dictates that the house's value grows as well.