if you are talking about rent and general cost of living. we have discussed it recently http://akihabara.tokyo/question/12379/whats-the-cost-of-living-in-japan/

as for the difference between big and small cities... There is a huge difference between Tokyo (and maybe Kyoto) and the rest of the country. The same apartment/house will cost twice as much in Tokyo as in a smaller city like Toyama (my only reference city, sorry :)).

For buying new ones, single houses are 20 million yen upwards and flats start around 25 million yen.

For rental. a one-room apartment in Tokyo can cost 80,000 yen if the location is close to the station. They get cheaper as you move to smaller cities and countryside. Landlords use key money, registration fees to make sure that they still get a large chunk of money when a new contract is made.