Butter and mayo are similar in the fact that they each contain fat. The fat in mayo keeps the sandwich from sticking while the protein creates the crusty golden brown goodness that the grilled cheese is known for. Generally mayo is actually better to use than butter for a more even and consistent browning. The fact that there are more proteins, which are more complex, in the egg will cause it to brown easier and more evenly with a better browned flavor quality. The reaction in question is called the Maillard reaction.
Fat from the butter will also brown and prevent the sandwich from sticking, but not with as good of a flavor quality or crispness that proteins in mayo provide.
Latest Answers