Most of the year, only the Imperial Palace East Gardens, with the remains of the former Tokugawa Shogunate’s castle and a nice Japanese garden inside, are open to the public. To get closer to the Emperor, be sure to apply in advance (from a month up to four days before) for a guided tour of the outer Imperial Palace buildings through the Imperial Household Agency on this website, though it may be Japanese-language only.
On January 2nd and December 23rd can you visit the inner Imperial Palace buildings and actually see the Emperor and family on the balcony, though you’ll be fighting with hordes of local Japanese tourists.