The oldest structures still standing in Japan are found at Horyu-ji in Nara. They were built in the Asuka period (the oldest is 7th century). Traditional Japanese structures are made of wood so many historical buildings have been burnt down, especially during WWII. Many of the temples, shrines and castles of Japan have been heavily renovated or completely reconstructed throughout history.
During the Meiji period western influences made their way into Japanese architecture. Good examples of this are Tokyo Station and the National Diet Building in Tokyo.
After WWII much of Japan needed to be reconstructed. In modern Japan there is a great gap between the construction of landmark buildings and normal housing or commercial buildings.
Most houses in Japan are still wooden and many still have Japanese features such as tatami flooring and paper interior walls or doors. The exterior of houses are sometimes in traditional Japanese styles as well. Even modern concrete and steal condominiums (called manshon in Japanese) often have one Japanese style room.
Manshons and houses in highly urbanized areas sometimes have very original designs. If you take a walk in expensive areas such as Minami or Setagaya in Tokyo you will see a high concentration of original architecture. As in most countries, in the suburbs, there is very little original architecture.
Many of Japan's landmark buildings have award winning architecture. A walk though the major business districts of Tokyo such as Shinjuku or Shiodome is recommended for those interested in modern architecture. Kyoto or Nara are the obvious choices if you are interested in traditional Japanese Architecture.