Many groups of young Kohanim gathered together with the keys to the Heichal [Sanctuary] in their hands. They ascended to the roof of the Heichal and called out, "Master of the World! Since we have not merited to be faithful treasurers the keys shall be transferred back to You!" They threw the keys heavenward and the form of a hand appeared and took the keys from them. The Kohanim then threw themselves from the roof into the burning remnants of the Temple below.The Heichal had but one gateway, so if the keys mentioned here were to that gateway why does the Gemara refer to them in the plural? The Gemara also implies that each of the many groups of Kohanim was holding one or more of the keys (otherwise it could have stated simply that "many young Kohanim gathered together...") — just how many keys did the gateway have? And why do the Kohanim refer to themselves as "treasurers" [גזברין]?
The term Heichal in its simplest sense refers to the chamber of the Sanctuary Building where the Menorah, Table of the Show Bread, and Incense Altar were housed. Also known as the Kodesh [Holy], this chamber was accessed by a gateway in its eastern wall leading in from the Antechamber. Like all Temple gateways, its double doors were secured with a keyed lock, but unlike all other gateways, it had two sets of doors. If we assume that every lock in the Temple was unique and required its own separate key, the Heichal gateway would thus require two keys. Below is an animation showing how the Heichal doors opened:
We find that the term Heichal is sometimes used to describe the entire Sanctuary Building (see Middos 4:6). If that is the case here then the "keys to the Heichal" would also include the keys used to unlock the doors of the tauim. [There were 38 tauim distributed over three levels. The first two levels had 5 tauim in the north, 5 in the south, and 3 in the west. The third level matched the other two except that there were only 2 in the west. For more information see this post.] Each tau connected to the tauim on either side of it and also to the one above it. Without working out the specifics of which of these openings had keyed locks and rather assuming simply that there was one key per tau gives us 38 more keys. Adding these to the four mentioned above yields about 42 keys. If each group of Kohanim in the Gemara's story was holding one key, this would certainly be enough to qualify as "many" groups. This approach would also explain why they refer to themselves as "treasurers" since the tauim were where the Temple's gold and silver was stored.
For further analysis of this Gemara see the explanation provided by the Kollel Iyun Hadaf.