The chief architectural remains, before the Gupta period, other than stupas and their surrounding gateways and railings, are artificial caves, excavated for religious purposes. Early specimens show a slavish imitation of carpentry that proves conclusively that the art of building in stone was still not fully developed. Two caves of Barabar Hill, 61.5 km. north of Buddhagaya, are in the form of a plain rectangular outer hall, at one end of which there is an inner chamber with a curved wall and overhanging caves. The caves were evidently substituted for a standardized religious meeting place consisting of a round thatched hut standing in a courtyard, and their designer could not transcend the pattern to which he had been used. Similar dependence on wooden models is evident in many other features of design until the Gupta period.

The caves of the Barabar and Nagarjuni Hills are unadorned, with the exception of one at Nagarjuni, near Barabar, which has a comparatively simple carved entrance, added during or soon after the Mauryan period. The inner walls of all the caves are finely polished, no doubt by workmen of the school that was responsible for the polish of the Asokan columns.

The Barabar caves, a fine specimen of workmanship needs to be added in the scheme of things for tourism development and if connected within the surroundings of Buddhagaya it sure will evoke much interest in the visitors.