The identity morphism (identity mapping) is called the trivial automorphism in some contexts. Respectively, other (non-identity) automorphisms are called nontrivial automorphisms.
The exact definition of an automorphism depends on the type of "mathematical object" in question and what, precisely, constitutes an "isomorphism" of that object. The most general setting in which these words have meaning is an abstract branch of mathematics called category theory. Category theory deals with abstract objects and morphisms between those objects.
In category theory, an automorphism is an endomorphism (i.e., a morphism from an object to itself) which is also an isomorphism (in the categorical sense of the word).
This is a very abstract definition since, in category theory, morphisms aren't necessarily functions and objects aren't necessarily sets. In most concrete settings, however, the objects will be sets with some additional structure and the morphisms will be functions preserving that structure.
By definition every isomorphism has an inverse which is also an isomorphism, and since the inverse is also an endomorphism of the same object it is an automorphism.
The automorphism group of an object X in a category C is denoted AutC(X), or simply Aut(X) if the category is clear from context.
In set theory, an arbitrary permutation of the elements of a set X is an automorphism. The automorphism group of X is also called the symmetric group on X.
In elementary arithmetic, the set of integers, Z, considered as a group under addition, has a unique nontrivial automorphism: negation. Considered as a ring, however, it has only the trivial automorphism. Generally speaking, negation is an automorphism of any abelian group, but not of a ring or field.
A group automorphism is a group isomorphism from a group to itself. Informally, it is a permutation of the group elements such that the structure remains unchanged. For every group G there is a natural group homomorphism G -> Aut(G) whose image is the group Inn(G) of inner automorphisms and whose kernel is the center of G. Thus, if G has trivial center it can be embedded into its own automorphism group.
In graph theory an automorphism of a graph is a permutation of the nodes that preserves edges and non-edges. In particular, if two nodes are joined by an edge, so are their images under the permutation.
In geometry, an automorphism may be called a motion of the space. Specialized terminology is also used:
An automorphism of a differentiable manifoldM is a diffeomorphism from M to itself. The automorphism group is sometimes denoted Diff(M).
In topology, morphisms between topological spaces are called continuous maps, and an automorphism of a topological space is a homeomorphism of the space to itself, or self-homeomorphism (see homeomorphism group). In this example it is not sufficient for a morphism to be bijective to be an isomorphism.
One of the earliest group automorphisms (automorphism of a group, not simply a group of automorphisms of points) was given by the Irish mathematician William Rowan Hamilton in 1856, in his icosian calculus, where he discovered an order two automorphism, writing:
so that is a new fifth root of unity, connected with the former fifth root by relations of perfect reciprocity.
Inner and outer automorphisms
In some categories--notably groups, rings, and Lie algebras--it is possible to separate automorphisms into two types, called "inner" and "outer" automorphisms.
In the case of groups, the inner automorphisms are the conjugations by the elements of the group itself. For each element a of a group G, conjugation by a is the operation given by (or a-1ga; usage varies). One can easily check that conjugation by a is a group automorphism. The inner automorphisms form a normal subgroup of Aut(G), denoted by Inn(G); this is called Goursat's lemma.