Though Zogan flourished in the Asuka period, it gradually fell into decline in the Heian period (794-1185). Just a few fine examples of Zogan from that period have been discovered, such as small inlays applied on the door stoppers of the Ho-o-do Hall of Byodo-in Temple in Uji, Kyoto.

Later in the Kamakura period (1185-1333), Zogan was revived for use on round sword guards. Generally, these sword guards were inscribed with the characters for “Yamashiro-sumi山城住” or “Heianjo-sumi 平安城住” as the name of the sword. Later around 1530, Fushimi-sumi Kaneie, the sword guard inlay master, founded his original method of creating decorative designs. His Zogan techniques depicting beautiful sceneries and people were outstandingly delicate and superb employing various metals in different colors such as gold, silver, reddish copper, and common copper.

In the early Edo period (1603-1868), there were two outstanding Zogan master families in the Nishijin area of Kyoto. One was the Umetada Family and the other, the Shoami Family. The Umetada Family started their business as swordsmiths before employing Zogan. A particularly noteworthy Zogan artist from the Umetada Family was a man named Myoju who served Hideyoshi Toyotomi, one of the most powerful samurai lords who ruled Japan in the late 16th century. Myoju excelled in flat Zogan decorated with fine lines called Rinzu and Sayagata. The Shoami Family started from the master of samurai armor manufacture and their most well-known Zogan master was named Masanori. Disciples of both the Zogan master families spread throughout Japan and worked for regional lords as Zogan craftsmen. In this way, the Kyoto-style Zogan skill prevailed all around Japan.

Until end of the Edo period, Zogan was mostly used for decorating valuable items, such as swords, only for samurai lords and imperial members. Eventually, however, the Zogan technique began to be used for other items such as charcoal braziers and pipes. When Japan experienced its drastic social restoration in the Meiji period (1868-1912), the government banned the general public from possessing swords in 1876, a law which almost ended the Zogan tradition in Japan. However, some ardent artists, such as Yasunosuke Shikata and Yasuyuki Namikawa, fled overseas and succeeded to continue the tradition of Japanese Zogan. As a result, traditional Japanese inlay works were highly valued in Western countries, and many items were exported due to their sublime beauty.