It is told, that the palace was added to the entire palace complex in 1792 AD by Prince Bahadur Saha , the younger son of Prthivi Narayan Saha after his return from northern India, where he spent a couple of years. He brought with him new building ideals, reflecting an Anglo-Indian style with Moghul influence. New to Nepal and the architecture of the Newars was the introduction of large windows with glass panes and shutters for the large hall on third floor level. This hall probably represents the first hall covering a double span in the Kathmandu Valley, anticipating the sapirations of Bhimsen Thapa, who built his Bagh Darbar at Kathmandu some 20 years later. New was also the introduction of a fire place in the hall, which was attached to the eastern wall.
Prince Bahadur Saha was acting as the regent for the them king Ranbahadru Saha (1778-79 and 1785-1797 AD) and is said to have been the governor for the area of Patan. Later the building become to be known under the name of Silakhana, that mean it served as the armoury of the army that was stationed in Patan.