Journal of Lie Theory Volume 9 (1999) 271–284 ◯^{C}1999 Heldermann Verlag

Algebraic Subgroups of Lie Groups

Dong Hoon Lee

Communicated by E. B. Vinberg

Abstract. In this work, we introduce the notion of algebraic subgroups of complex Lie groups, and prove that every faithfully representable complex analytic group G admits an algebraic subgroup T (G) which is the largest in the sense that it contains all algebraic subgroups of G. Moreover, the rational representations of the algebraic subgroup T (G) are exactly the restrictions to T (G) of all complex analytic representations of G. This enables us to single out a certain subgroup of a faithfully representable real analytic group G with which the Tannaka duality theorem is restated.

Let K be a complex Lie subgroup of a complex analytic group G. We are first concerned with the question of when K admits the structure of an affine algebraic group which is compatible with the analytic structure of the ambient group G in the sense that the restriction to K of every complex analytic representation of G is rational. In [5], Hochschild and Mostow studied this question for the entire group G, i.e., K = G, and gave a complete characterization of such G, when it is a faithfully representable complex analytic group. In this work we introduce the notion of algebraic subgroups of a complex analytic group, and prove that every complex analytic representation of the ambient group is rational, when restricted to such an algebraic subgroup. Our first result (Theorem 2.8) states that every faithfully representable complex analytic group G always admits an algebraic subgroup T (G), which is the largest in the sense that it contains all algebraic subgroups of G, and that the rational representations of the algebraic subgroup T (G) are exactly the restriction to T (G) of all complex analytic representations of G. The subgroup T (G) was studied in [8] where the compatibility of its algebraic group structure with the analytic group structure of G was not fully established. Next we turn to real Lie groups. Chevalley’s complex formulation of Tannaka’s duality theorem states that if G is a compact real Lie group, then the universal complexification G^{+} of G is isomorphic with the group A(G) of all proper automorphisms of the algebra R(G) of representative functions of G. Harish-Chandra [2] established this result for semismple analytic groups, and Hochschild and Mostow gave several conditions, each of which characterizes those Lie groups G with finitely many components having such duality. One of these

ISSN 0949–5932 / $2.50 ◯^{C} Heldermann Verlag conditions states that G^{+} is equipped with the structure of an affine algebraic group in such a way that every complex analytic representation of G^{+} becomes a rational representation (or equivalently, every real analytic representation of G is induced by a rational representation of G^{+}.) We prove (Theorem 3.4) that every faithfully representable real analytic group G contains a closed normal analytic subgroup T (G) of G, which we call the Tannaka subgroup of G, such that T (G)^{+} is the maximal algebraic subgroup of G^{+}, and that every real analytic representation of T (G) is induced by a rational representation of T (G)^{+}.

The main body of this work was done while I was visiting TU Darmstadt, Germany for the summer term of 1997 under the sponsorship of Deutsche Forschungsgemeinschaft.

Convention and Terminology. Throughout this work, we will adopt the following convention. All representations of Lie groups are finite-dimensional and over the field ℂ of complex numbers. If a certain concept or result is valid for both real and complex Lie groups, we will simply use the expression Lie groups (thereby suppressing the words real or complex) with understanding that the concept or result is valid within the category of real Lie groups as well as of complex Lie groups. Thus for example the expression like an analytic representation of a Lie group means a real (resp. complex) analytic representation of a real (resp. complex) Lie group.

1. Preliminaries

In this section we put together some of basic results on Lie groups and their representative functions. For a detailed discussion we refer to [4] and [5]. Let G be a Lie group, and let R(G) denote the ℂ-algebra of analytic representative functions of G. If ϕ is an analytic representation of G, let [ϕ] denote the ℂ-linear space of all representative functions associated with ϕ. Then R(G) = ∪_{ϕ}[ϕ]. This is a fully stable ℂ-algebra in the sense that it is stable under the left and right translations by the elements of G and the involution f ↦ f′, where f′(x) = f(x^{−1}).

Let Q be any fully stable subalgebra of R(G). We regard Q as a Galgebra with G acting on Q by right translations. A G-algebra automorphism of Q is called a proper automorphism of Q. Thus a proper automorphism of Q is an algebra automorphism which commutes with the right translations f ↦ f·x : Q → Q for all x ∈ G. Let Aut_{G}(Q) denote the group of all proper automorphisms of Q. In a natural way, Aut_{G}(Q) is equipped with the structure of a pro-affine algebraic group, and it may be identified with the set Spec(Q, ℂ) of all specializations (i.e., ℂ-algebra homomorphisms) of Q into ℂ. The algebra Q is the polynomial algebra of the pro-affine algebraic group Aut_{G}(Q), where one views the elements f of Q as functions on Aut_{G}(Q) by the formula

f(α) = α(f)(1)

for α ∈ Aut_{G}(Q). For y ∈ G, the left translation τ_{y}: f ↦ y·f is a proper automorphism of Q, and hence y ↦ τ_{y} defines a canonical homomorphism τ: G → Aut_{G}(Q). If Q is finitely generated, then Aut_{G}(Q) has the usual structure of an affine algebraic group.

For a closed normal Lie subgroup M of G, let R(G, M) denote the subalgebra of R(G) consisting of all analytic representative functions associated with M -unipotent analytic representations of G (i.e., representations which are unipotent on M). R(G, M) is a fully stable subalgebra of R(G).

Universal algebraic hull. For any complex analytic group G, let A(G) = Aut_{G}(R(G)). The group A(G) acquires the structure of an irreducible pro-affine algebraic group over ℂ, for which R(G) is the polynomial algebra of A(G). We call A(G) the universal algebraic hull of G.

The term universal is justified by the following universal property of the canonical homomorphism τ: G → A(G).

Proposition 1.1. For any analytic representation ρ: G → GL(V, ℂ) there exists a unique rational representation ρ̃: A(G) → GL(V, ℂ) such that ρ̃ ∘ τ = ρ.

Representation radical and reductive subgroups. For any Lie group G, let N(G) denote the intersection of all kernels of semisimple analytic representations of G. The restriction to N(G) of every analytic representation of G is unipotent, and it may be characterized as the largest subgroup of G with this property. The subgroup N(G) is called the representation radical of G. If G is a faithfully representable analytic group, then N(G) is equal to the radical of the commutator subgroup of G. It follows from the theorem of Lie that every analytic representation of G is N(G)-unipotent, and hence, in particular, we have R(G) = R(G, N(G)).

A faithfully representable complex (resp. real) Lie group G is called reductive if every complex (resp. real) analytic representation of G is semisimple.

If G is a faithfully representable (real or complex) analytic group, it is a semi-direct product G = S·H, where S is a simply connected closed solvable normal subgroup and H is a reductive analytic subgroup of G. The subgroup S is called a nucleus of G.

Universal Complexification of Real Lie Groups. By a universal complexification of a real analytic group G, we mean a pair (G^{+}, γ), where G^{+} is a complex analytic group, and γ: G → G^{+} is a continuous homomorphism satisfying the following universal property:

For any complex analytic group K and any continuous homomorphism u: G → K, there exists a unique complex analytic homomorphism u^{+}: G^{+} → K such that u = u^{+} ∘ γ. The pair (G^{+}, γ) is uniquely determined (up to an equivalence) by this property.

If a real analytic group G admits a finite-dimensional faithful analytic representation, then the canonical map γ: G → G^{+} is injective, and the Lie algebra γ(G) is a real form of the Lie algebra of G^{+}. For the discussion of the universal complexification of non-connected Lie groups, see [4].

Conjugacy of reductive groups. For later use we establish here the conjugacy of reductive subgroups. The conjugacy of maximal reductive subgroups in a real or complex analytic group seems to be well-known although we are unable to find an explicit reference for it at least in the real case. As for the complex case, see [5], p. 95, where the conjugacy for reductive subgroups was proven using the structure theorem of complex reductive groups together with the known conjugacy of compact subgroups. Our proof is parallel to the usual proof of conjugacy theorem of reductive algebraic subgroups in linear algebraic groups. We begin with the following well-known result, whose proof is provided here for completeness.

Lemma 1.2. Let G be a reductive analytic group. For any finite-dimensional analytic G-module V , every analytic 1-cocyle of G in V is a coboundary.

Proof. Let F denote either ℝ or ℂ, depending on whether G is a real or complex analytic group. Let f: G → V be a 1-cocycle, and we define an action of G on the direct sum V ⊕ F by

x·(v, a) = (af(x) + x·v, a)

where x ∈ G, v ∈ V and a ∈ F. The 1-cocycle identity implies that the above defines an analytic G-module structure on V ⊕ F, which contains V as a Gsubmodule. Since G is reductive, there is a G-module complement for V in V ⊕ F. There is an element v ∈ V such that (v, 1) spans this complement over F. Since the reductive group G acts trivially on any 1-dimensional G-module, we have f(x) + x·v = v for all x ∈ G, proving that f is a coboundary.

Proposition 1.3. Let G be a faithfully representable (real or complex) analytic group, and write G in the form

G = K·P

(semi-direct product), where K is a nucleus of G and P a closed reductive analytic subgroup of G. If Q is any reductive subgroup of G, then there exists an element u ∈ N(G) such that uQu^{−1} ⊂ P. In particular, P is a maximal reductive subgroups, and any two maximal reductive subgroups are conjugate by an element of N(G).

Proof. We prove the assertion by induction on dim(G). We put N = N(G). Assume N = (1). Since N is the radical of the commutator subgroup [G, G], we have N = [G, R], where R is the radical of G, and hence N = (1) implies that R is central in G. Since the nucleus K is contained in R, the semidirect product G = K·P is a direct product. Let η denote the restriction to Q of the projection map G = K·P → K. Then η(Q) is a reductive subgroup of the vector group K, and hence must be trivial. This shows that Q ⊂ P, proving our assertion in this case. Thus we may assume that N ≠ (1). The center Z of N is a normal (vector) subgroup of G, and if π: G → G/N denotes the canonical morphism, then by induction hypothesis there exists an element u′ ∈ N(G/Z) = N(G)/Z such that u′π(Q)(u′)^{−1} ⊂ π(P). This shows that uQu^{−1} ⊂ ZP for some u ∈ N(G), and replacing uQu^{−1} by Q, if necessary, we may assume that Q ⊂ ZP. Noting that the subgroup ZP is a semidirect product, let ζ: Q → Z and π: Q → P denote the projections (restricted to Q) of ZP onto Z and P, respectively, so that x = ζ(x)π(x). We view the vector group Z as an analytic Q-module, where Q acts on Z by x·z = π(x)zπ(x)^{−1}. Then ζ(xy) = ζ(x) + x·ζ(y) for all x, y ∈ Q, i.e. ζ is a cocycle of Q with values in Z. By the lemma above, it is a coboundary, and hence there exists an element v ∈ Z such that

ζ(x) = v − x·v

for all x ∈ Q. Thus for x ∈ Q, xπ(x)^{−1} = ζ(x) = vπ(x)v^{−1}π(x)^{−1}, and this implies v^{−1}xv = π(x) ∈ P, proving v^{−1}Qv ⊂ P.

Decomposition of representative algebras. Let G be a complex Lie group and assume G = A·B (semidirect product), where A and B are closed complex Lie subgroups of G with B normal in G. Here we develop a condition for the extendability of a representation of B to the entire group G.

For any function f: B → ℂ (resp. f: A → ℂ), define f^{+}: G → ℂ by f^{+}(ab) = f(b) (resp. f^{+}(ab) = f(a)) for a ∈ A and b ∈ B. For a ∈ A, κ(a) denotes the automorphism of B given by b ↦ a^{−1}ba.

Lemma 1.4. If K is a closed normal Lie subgroup of G such that K ⊂ B, then for g ∈ R(B, K), g ∘ κ(A) ⊂ R(B, K).

Proof. For a ∈ A and b ∈ B, we have

(b·g) ∘ κ(a) = (aba^{−1})·(g ∘ κ(a))

For a fixed a and with b ranges over B, this shows that g ∘ κ(a) ∈ R(B). To show g ∘ κ(a) ⊂ R(B, K), let V and V′ be the finite-dimensional subspaces that are spanned by B·g and B·(g ∘ κ(a)), respectively, and let ρ and ρ′ be the representations of B on V and V′, respectively, by left translations. We have ρ′(z) = θ ∘ ρ(aza^{−1}) ∘ θ^{−1} for z ∈ B, where θ: V → V′ denotes the linear isomorphism f ↦ f ∘ κ(a). Hence, for any z ∈ B, and for any positive integer m, we have

(ρ′(z) − 1)^{m} = θ ∘ (ρ(aza^{−1}) − 1)^{m} ∘ θ^{−1}

Since g ∈ R(B, K), ρ(K) acts on V unipotently, and the above equality shows that ρ′(K) acts on V′ unipotently, proving g ∘ κ(a) ∈ R(B, K).

The proof of the following lemma is essentially the same as that of [4], Propoposition 2.4).

Lemma 1.5. Let G = A·B be a semidirect product of analytic groups with B normal in G, and let K be a closed normal analytic subgroup of G with K ⊂ B. Let R(G, K)_{B} denote the image of the restriction map R(G, K) → R(B, K). Then for g ∈ R(B, K), the following are equivalent:

(i) g ∈ R(G, K)_{B}; (ii) g ∘ κ(A) = {g ∘ κ(a): a ∈ A} spans a finite-dimensional subspace of R(B, K); (iii) g^{+} ∈ R(G, K). Moreover, (f, g) ↦ f^{+}·g^{+} induces an isomorphism

R(A) ⊗ R(G, K)_{B} ≅ R(G, K)

In particular, R(G, K) ≅ R(A) ⊗ R(B, K) canonically if and only if the restriction morphism R(G, K) → R(B, K) is surjective.

2. Algebraic Subgroups of Analytic Groups

A complex Lie subgroup K of a faithfully representable complex analytic group G is called an algebraic subgroup of G if the restriction algebra R(G)_{K} (i.e., the restriction to K of the functions in R(G)) is finitely generated as a subalgebra of R(K), and if K is an affine algebraic group with R(G)_{K} as its polynomial algebra. Remarks. (1) From the definition, it follows that a closed complex Lie subgroup K of G is an algebraic subgroup if and only if the following two conditions are satisfied:

(i) R(G)_{K} is finitely generated;

(ii) The canonical map K → Aut_{K}(R(G)_{K}) is an isomorphism of groups.

(2) For any algebraic subgroup K of G, the algebraic group structure of K is determined not by the analytic structure of K itself, but entirely by the analytic structure of the ambient group G.

(3) We note that an algebraic subgroup of a faithfully representable complex analytic group G is necessarily closed. In fact, if ρ: G → GL(V, ℂ) is a faithful complex analytic representation, then ρ(K) is an algebraic subgroup of GL(V, ℂ). Thus ρ(K) is closed in ρ(G), and hence K is closed in G.

In the sequel, P (H), for any affine (or pro-affine) algebraic group H, denotes the polynomial algebra of H.

Proposition 2.1. Let K be a closed complex Lie subgroup of a faithfully representable complex analytic group G. The following are equivalent.

(i) K is an algebraic subgroup of G; (ii) If τ: G → A(G) denotes the canonical injection, τ(K) is an (affine) algebraic subgroup of the pro-affine algebraic group A(G).

If K is an algebraic subgroup of G, then the restriction to K of every complex analytic representation of G is rational.

Proof. (i) ⟹(ii): We need to show that if g ∈ P (A(G)) = R(G), then g ∘ τ_{K} ∈ P (K). Write g as g = ε ∘ ψ, where ψ: A(G) → GL(V, ℂ) is a rational representation, and ε: End(V) → ℂ is a linear functional. Then ψ ∘ τ: G → GL(V, ℂ) is a complex analytic representation, and hence ε ∘ (ψ ∘ τ) ∈ R(G). Thus g ∘ τ_{K} = (g ∘ τ)_{K} = (ε ∘ ψ ∘ τ)_{K} ∈ R(G)_{K} = P (K) follows.

(ii)⟹(i): Since τ(K) is an (affine) algebraic subgroup of A(G), P (τ(K)) is finitely generated and we have P (τ(K)) = P (A(G))_{τ(K)} = R(G)_{τ(K)}. Transporting the affine algebraic group structure of τ(K) to K using the group isomorphism τ_{K}, we obtain an affine algebraic group structure on K so that P (K) = R(G)_{K}.

For the second assertion, let ρ: G → GL(V, ℂ) be a complex analytic representation. We must show that if η is any ℂ-linear functional on the linear space End_{ℂ}(V), then η ∘ ρ_{K} ∈ P (K). Indeed, η ∘ ρ ∈ R(G), and hence η ∘ ρ_{K} = (η ∘ ρ)_{K} ∈ R(G)_{K} = P (K).

Lemma 2.2. If K is a reductive complex Lie subgroup of a faithfully representable complex analytic group G such that K is finite modulo its identity component K_{0}, then K is an algebraic subgroup of G.

Proof. Let H be a maximal reductive complex Lie subgroup of G that contains K. Then H is connected, and G is a semidirect product G = S·H, where S is a nucleus of G. We first show that H is an algebraic subgroup of G. Every complex analytic representation of H extends in an obvious way to a complex analytic representation of G, and hence it follows that the restriction map R(G) → R(H) is surjective, i.e., we have R(H) = R(G)_{H}. By ([5], Th. 5.2), R(H) is finitely generated, and the canonical monomorphism

H → A(H) = Aut_{H}(R(H))

is an isomorphism. This shows that H is an algebraic subgroup of G. Next we show K is an algebraic subgroup of G. Let Q be a maximal compact subgroup of K. Then K may be identified with the universal complexification of Q. (This is Theorem 3.2 of [5], when K itself is connected. For non-connected K, one may prove the assertion by verifying directly the universal property defining the universal complexification, making use of the facts that K = K_{0}Q and Q_{0} = Q ∩ K_{0}, where L_{0} for any Lie group L denotes the connected component of L containing the identity element.) It follows from ([4], Theorem 9.5 and Theorem 11.1) that R(Q) = R(K) is finitely generated and that K = Q^{+} is identified with an affine algebraic group with its polynomial algebra R(K) in such a way that every complex analytic representation of K becomes rational. It remains to show that R(K) = R(G)_{K}. If we view the affine algebraic group H as a linear algebraic group and the inclusion map j: K → H as a linear representation, then j is rational, and hence K = j(K) is Zariski closed in the algebraic group H. This shows that R(H) → R(K) is surjective. Since we have already observed that R(H) = R(G)_{H}, we see that R(K) = R(G)_{K}.

Now we turn our attention to unipotent subgroups of complex linear groups. Let N be a unipotent complex analytic subgroup of a full linear group GL(W, ℂ). There is a natural algebraic group structure on the simply connected nilpotent complex analytic group N. If ρ: N → GL(U, ℂ) is a unipotent complex analytic representation of N, then ρ is a polynomial map of degree ≤ mn, where m = dim W, and n = dim U. In fact, for x ∈ N, we have

ρ(x) = ρ(exp_{W} ∘ log_{W} (x)) = exp_{U} ∘dρ(log_{W} (x))  (1)

where exp_{W} and exp_{U} denote exponential maps on the full linear groups GL(W, ℂ) and GL(U, ℂ), respectively.

Since N is unipotent, 1 − x is a nilpotent linear transformation on W, and we have

log_{W} (x) = − ∑_{i=1}^{m} \frac{(1 − x)^{i}}{i}  (2)

Let N be the Lie algebra of N. Since every element z of dρ(N) is a nilpotent linear transformation of U, we have

exp_{U}(z) = ∑_{i=0}^{n} \frac{1}{i!}z^{i}  (3)

It follows from (1), (2) and (3) that ρ is a polynomial map of degree ≤ mn.

Lemma 2.3. Let N be as above. Then P (N) = R(N, N) (i.e., the polynomial algebra of the algebraic group N coincides with the algebra R(N, N) consisting of all representative functions of N associated with the unipotent analytic representations of N.)

Proof. Clearly P (N) ⊂ R(N, N). To show P (N) ⊃ R(N, N), let f ∈ R(N, N), and express f = λ ∘ ρ, where ρ: N → GL(U, ℂ) is a unipotent complex analytic representation, and where λ is a ℂ-linear functional on End(U, ℂ). By what we have observed above, ρ is a polynomial map, and hence f = λ ∘ ρ is a polynomial function, proving f ∈ P (N).

Lemma 2.4. Let N be a unipotent complex analytic subgroup of a full complex linear group GL(V, ℂ), and let H be a complex Lie group. Suppose there is a complex analytic homomorphism κ: H → Aut(N). Then for each f ∈ R(N, N), the set {f ∘ κ(h) : h ∈ H} spans a finite-dimensional subspace of R(N, N).

Proof. By Lemma 1.4, f ∘κ(H) ⊂ R(N, N). Let h ∈ H, and consider κ(h): N → N. By the lemma above, each κ(h): N → N is a polynomial map of degree ≤ m^{2}, where m is the dimension of the space on which N operates. Thus if f is a polynomial function on N of degree ≤ n, then f ∘ κ(h) is a polynomial function of degree ≤ nm^{2}. This shows that the set f ∘ κ(H) spans a finite-dimensional space.

The following is ([6], Lemma 2.1).

Lemma 2.5. Let L be a finite-dimensional Lie algebra over an infinite field, that is a semidirect sum L = H + K, where K is a solvable ideal of G, and H is a complementary subalgebra that is reductive in L. Let M = [L, K]. Then K contains a nilpotent subalgebra P such that K = P+M (not necessarily semidirect sum) and that P centralizes H.

Let L be a subgroup of a group G, and let ρ be a representation of L in a ℂ-linear space V . A representation σ of G is said to be an extension of ρ (or simply ρ is extendable to σ) if the representation space of σ contains V as a σ(L)-invariant subspace and σ(x) and ρ(x) coincides in V for all x ∈ L. It is easy to verify that if every analytic representation of a closed Lie subgroup L of a Lie group G is extendable to an analytic representation of L, then the restriction map R(G) → R(L) is surjective.

Lemma 2.6. Let G be a faithfully representable analytic group G, and let N denote the representation radical of G. For any maximal reductive subgroup H of G, every analytic representation of HN is extendable to an N -unipotent analytic representation of G. In particular, the restriction map

R(G) = R(G, N) → R(HN, N)

is surjective.

Proof. Write the faithfully representable group G as a semidirect product G = K·H, where K is a nucleus of G. By Lemma 2.5, K contains a simply connected nilpotent analytic subgroup P such that K = P N and that P centralizes H. Then G = P NH, and clearly we have [P, G] ⊂ N. Since G/HN is a vector group and G = P HN, we can find one-parameter subgroups P_{1}, …, P_{r} ⊂ P such that G is a successive semidirect product G = P_{r} … P_{1}(HN). Let D_{0} = HN, and define D_{i+1} = P_{i+1}D_{i}, i = 0, 1, 2, …, r − 1. We show that every N -unipotent analytic representation ρ of D_{i} (0 ≤ i ≤ r − 1) is extendable to an N -unipotent analytic representation σ of D_{i+1} (i.e., the representation space W of σ contains the representation space V of ρ as a D_{i}-stable subspace and σ coincides with ρ in V .) Since [P, G] ⊂ N, the action of the group P_{i+1} on D_{i}/N is trivial. Hence applying [3], Th. 2.2 of Chap. 18) to the semidirect product D_{i+1} = P_{i+1}·D_{i}, we see that any N -unipotent analytic representation ρ of D_{i} extendable to an N - unipotent analytic representation σ of D_{i+1}. Putting these extensions together, we see that every analytic representation of HN is extendable to an N -unipotent analytic representation of G.

The second assertion follows from the remark preceding the lemma.

Lemma 2.7. Let G be a faithfully representable complex analytic group. Then N(G) is a unipotent algebraic subgroup of G.

Proof. Let τ: G → GL(W, ℂ) be faithful complex analytic representation of G, and put N = N(G). Then N = rad(G′) is a simply connected nilpotent analytic group, and hence N is isomorphic with the unipotent algebraic subgroup τ(N) of GL(W, ℂ). τ induces an isomorphism of ℂ-algebras

R(N, N) ≅ R(τ(N), τ(N))

and R(τ(N), τ(N)) is the polynomial algebra of the algebraic group τ(N) by Corollary 2.3. This, in particular, shows that R(N, N)is finitely generated. That the canonical map

N → Aut_{N}(R(N, N))

is an isomorphism follows from the isomorphisms

N ≅ τ(N) ≅ Aut_{τ(N)}(R(τ(N), τ(N))) ≅ Aut_{N}(R(N, N))

Now it remains to show R(N, N) = R(G)_{N}. By Lemma 2.6 the restriction R(G) → R(NH, N) is surjective, and hence it is sufficient to show that the restriction map R(NH, N) → R(N, N) is surjective. But this follows from Lemma 1.5 and Lemma 2.4.

Theorem 2.8. Let G be a faithfully representable complex analytic group and let H be a maximal reductive complex analytic subgroup of G. The complex analytic normal subgroup HN(G), which is independent of maximal reductive subgroups H, is the maximal algebraic subgroup of G in the sense that it contains all algebraic subgroups of G. Any complex analytic representation of G induces a rational representation of HN(G), and conversely every rational representation of HN(G) is obtained in this way.

Proof. Let N = N(G) and let τ: G → A(G) denote the canonical injection. H and N are algebraic subgroups of G by Lemma 2.2 and Lemma 2.7. Hence τ(N) and τ(H) are algebraic subgroups of the pro-affine algebraic group A(G) in the usual sense by Proposition 2.1. It follows that τ(NH) = τ(N)τ(H) is Zariski closed in the pro-affine algebraic group A(G), proving that NH is an algebraic subgroup of G. That the subgroup HN is independent of maximal reductive subgroups H follows from Proposition 1.3. We now show that HN is the maximal algebraic subgroup of G. Let D be any algebraic subgroup of G. We claim D ⊂ HN. Suppose D is not contained in HN. We first show that there is a complex analytic representation φ of G such that φ(D) is a nontrivial unipotent subgroup. Let π: G → G/HN be the canonical morphism, and let

ρ: G → GL(V, ℂ)

be a faithful complex analytic representation of G. The image ρ(HN) is an algebraic subgroup of GL(V, ℂ) by Proposition 2.1. Since HN is normal in G, the algebraic subgroup ρ(HN) of GL(V, ℂ) is normal in ρ(G) and hence also in the Zariski closure ρ(G)^{#} of ρ(G) in GL(V, ℂ). Thus ρ(G)^{#}/ρ(HN) is a linear algebraic group, and the composite map

G/HN → ρ(G)/ρ(HN) ⟶^{⊆} ρ(G)^{#}/ρ(HN)

defines a faithful complex analytic representation ρ̃ of G/NH. Let φ = ρ̃∘π. Since D is algebraic in G, φ(D) is Zariski closed in the algebraic group ρ(G)^{#}/ρ(HN) by Proposition 2.1. Since π(D) is a nontrivial subgroup of the vector group G/HN with finitely many connected components, π(D) is a nontrivial vector subgroup of G/HN. Thus φ(D) = ρ̃(π(D)) is a complex vector subgroup of the abelian algebraic group ρ̃(π(G))^{#}, and hence the algebraic subgroup φ(D) is unipotent. Next we show that there is a nontrivial semisimple analytic representation ψ of G. Choose a faithful, semisimple complex analytic representation σ of the complex vector group G/HN. Then σ(π(D)) ≠ {1}, and we take ψ to be σ ∘ π. If γ denotes the direct sum of the representations φ and ψ, then γ is a complex analytic representation of G such that γ(D) is not Zariski closed, and this contradicts the assumption that D is an algebraic subgroup of G. This shows that D ⊂ HN.

We now prove the second assertion. Let ρ be a rational representation of the algebraic group HN. Since N is a unipotent algebraic subgroup of HN, ρ is an N -unipotent analytic representation of the analytic group HN, and Lemma 2.6 enables us to extend ρ to a complex analytic representation of G. Notation. For a faithfully representable complex analytic group G, the maximal algebraic subgroup HN of G in Theorem 2.8 is denoted by T (G).

3. Tannaka subgroup of real Lie groups

Lemma 3.1. Let G be a faithfully representable real analytic group. Then N(G)^{+} = N(G^{+}).

Proof. Put N = N(G), N^{+} = N(G)^{+}, and let G, G^{+}, N and N^{+} denote the Lie algebras of G, G^{+}, N and N^{+}, respectively. Since an analytic representation ρ: G → GL(V, ℂ) is semisimple if and only if the corresponding complex analytic representation ρ^{+}: G^{+} → GL(V, C) is semisimple, it follows from the definition of the representation radical that the canonical injection γ: G → G^{+}maps N into N(G^{+}). Thus there is a unique complex analytic homomorphism γ′: N^{+} → N(G^{+}) such that γ′ ∘ η = γ_{N}, where η: N → N^{+} denotes the canonical injection. We claim that γ′ is an isomorphism. We first note

dim_{ℂ} N(G^{+}) = dim_{ℂ} N^{+} = dim_{ℝ} N

In fact, we have the equality:

L(N(G^{+})) = Rad([G^{+}, G^{+}]) = Rad(ℂ ⊗ [G, G]) = ℂ ⊗ Rad([G, G]) = ℂ ⊗ N where Rad(L) for any Lie algebra L denotes the radical of L. (Here we identified G with its image dγ(G) in G^{+}, which is a real form of the complex Lie algebra G^{+}.) This equality shows that dγ(N) spans L(N(G^{+})) over ℂ. Since dη(N) is a real form of N^{+}, it follows from dγ′ ∘ dη = dγ_{N} that dγ′ is surjective, and hence is an isomorphism due to the equality dim_{ℂ} N(G^{+}) = dim_{ℂ} N^{+}. This means that γ′ is a covering morphism of N(G^{+}). But N(G^{+}) is simply connected, and so γ′ must be an isomorphism, proving our assertion.

Before we state our main result in this section, we prove the following general lemma:

Lemma 3.2. If G = K ×_{ϕ}H is a semidirect product of real analytic groups K and H with respect to an analytic action ϕ: H → Aut(K), then G^{+} is the semidirect product G^{+} = K^{+} ×_{ϕ^{+}} H^{+}, where ϕ^{+}: H^{+} → Aut(K^{+}) is the homomorphism defined by the commutative diagram

H^{+} ⟶^{ϕ^{+}} Aut(K^{+})
↑  ↑^{id}
H ⟶_{ϕ′} Aut(K^{+})

and where ϕ′: H → Aut(K^{+}) is the unique homomorphism satisfying the commutative diagram

K^{+} ⟶^{ϕ′(h)} K^{+}
↑  ↑
K ⟶_{ϕ(h)} K.

Proof. The map γ: K ×_{ϕ} H → K^{+} ×_{ϕ^{+}} H^{+} defined by γ(k, h) = (γ_{K}(k), γ_{H}(h)) is easily seen to be a homomorphism. We claim that K^{+} ×_{ϕ^{+}} H^{+} together with the map γ satisfies the universal property characterizing the universal complexification of K ×_{ϕ} H. Let α: K ×_{ϕ} H → L be a real analytic homomorphism into a complex analytic group L, and define

α^{+}: K^{+} ×_{ϕ^{+}} H^{+} → L

by α^{+}(k′, h′) = α_{K}^{+}(k′)α_{H}^{+}(h′). In proving that α^{+} is a homomorphism, we must show that

α^{+}((k′_{1}, h′_{1})(k′_{2}, h′_{2})) = α^{+}(k_{1′}, h′_{1})α^{+}(k′_{2}, h′_{2})

or equivalently that

α_{K}^{+}(ϕ^{+}(h′)(k′)) = α_{H}^{+}(h′)α_{K}^{+}(k′)α_{H}^{+}(h′)^{−1}  (4)

for h′ ∈ H^{+} and k′ ∈ K^{+}. Since α: K ×_{ϕ} H → L is a homomorphism, we have

α_{K}(ϕ(h)(k)) = α_{H}(h)α_{K}(k)α_{H}(h)^{−1}  (5)

Define ϕ̂^{+}: H^{+} × K^{+} → K^{+}, and ϕ̂: H × K → K by ϕ̂^{+}(h′, k′) = ϕ^{+}(h′)(k′), and ϕ̂(h, k) = ϕ(h)(k). Then the equations (4) and (5) are equivalent to the commutativity of the diagrams:

\begin{matrix}H^{+} × K^{+} & \underrightarrow{_{ϕ^{+}}} & K^{+} \\ ^{α_{H}^{+}×α_{K}^{+}} ↓ & & ↓^{α_{K}^{+}} \\ L × L & ⟶_{η} & L \end{matrix}  (6)

and

\begin{matrix}H × K & \underrightarrow{_{ϕ}} & K \\ ^{α_{H}×α_{K}} ↓ & & ↓^{α_{K}} \\ L × L & ⟶_{η} & L \end{matrix}  (7)

respectively, where η: L × L → L is given by η(x, y) = xyx^{−1}. The commutativity of the diagrams (6) and (7) holds if and only if their respective Lie algebra diagrams (8) and (9) commute:

\begin{matrix}H^{+} × K^{+} & ⟶^{d _{ϕ^{+}}} & K^{+} \\ ^{dα_{H}^{+}×dα_{K}^{+}} ↓ & & ↓^{dα_{K}} \\ L × L & ⟶_{dη} & L \end{matrix}  (8)

and

\begin{matrix}H × K & ⟶^{d _{ϕ}} & K \\ ^{dα×dα}↓ & & ↓^{dα} \\ L × L & ⟶_{dη} & L \end{matrix}  (9)

where H^{+} and K^{+} denote the Lie algebras of H^{+} and K^{+}, respectively. Since the Lie algebras in the diagram (9) span the corresponding complex Lie algebras in (8) over C, the commutativity of (8) follows from that of (9), and consequently the diagram (6) commutes, proving (4).

That α^{+} is the only morphism with the relation α = α^{+} ∘ γ follows from the fact that Im(dγ) spans K^{+} ×_{dϕ^{+}} H^{+} = L(K^{+} ×_{ϕ^{+}} H^{+}).

Lemma 3.3. If G is a faithfully representable real analytic group, then so is its universal complexification.

Proof. Write the faithfully representable G as a semidirect product G = S·H, where S is a nucleus of G and H a maximal reductive subgroup of G. The universal complexification S^{+} of the simply connected solvable real analytic group S is a simply connected solvable complex analytic group, and the universal complexification H^{+} of the real reductive analytic subgroup H is a reductive complex analytic group. Hence by Lemma 3.2, we have a semidirect product G^{+} = S^{+}·H^{+}, and consequently G^{+} is faithfully representable by ([3], Theorem 4.3, p. 223).

For a faithfully representable real analytic group G, set T (G) = HN(G), where H is a maximal reductive subgroup of G. By the conjugacy of reductive subgroups (Proposition 1.3), T (G) is independent of choice of a maximal reductive subgroup H. We call T (G) the Tannaka subgroup of G.

Now we are ready to state and prove our main result in this section.

Theorem 3.4. Let G be any faithfully representable real analytic group. Then

(i) T (G) is a closed normal analytic subgroup of G, and T (G)^{+} = T (G^{+}), i.e., T (G)^{+} is the maximal algebraic subgroup of the complex analytic group G^{+};

(ii) Every analytic representation of T (G) is induced by a rational representation of T (G)^{+}.

Proof. Let γ: G → G^{+} be the canonical injection. By Lemma 3.1 and Lemma 3.2, we have T (G)^{+} = (HN(G))^{+} ≅ H^{+}N(G)^{+} (semidirect product). Since H^{+} is a maximal reductive subgroup of G^{+}, it follows that T (G)^{+} is the maximal algebraic subgroup of G^{+}, proving (i). To prove (ii), let ρ: T (G) → GL(V, ℂ) be an analytic representation. By Lemma 2.6, ρ extends to an analytic representation σ: G → GL(W, ℂ), and the induced complex analytic representation σ^{+}: G^{+} → GL(W, ℂ) becomes a rational representation, when restricted to the maximal algebraic subgroup T (G^{+}) of G^{+} (Theorem 2.8). The representation space V of ρ is a σ(T (G))-invariant and hence also σ^{+}(T (G)^{+})-invariant subspace of W, and the representation

t′ ⟼ σ^{+}(t′)_{V} : T (G)^{+} → GL(V, ℂ)

is exactly the complex analytic representation ρ^{+} of T (G)^{+} that is induced by ρ. Since T (G)^{+} = T (G^{+}) by (i), and since σ_{T (G)^{+}}^{+} is a rational representation of T (G^{+}), it follows that ρ^{+} is a rational representation of T (G)^{+}.

References

[1]  Chevalley, C., “Theory of Lie groups,” Princeton University Press, 1946.

[2] Harish-Chandra, Lie algebras and the Tannaka duality theorem, Ann. of Math. 51 (1950), 299–330.

[3] Hochschild, G., “The structure of Lie groups,” Holden-Day, San Francisko, 1965.

[4] Hochschild, G., and G. D. Mostow, Representations and representative functions of Lie groups, Ann. of Math. 66 (1957), 495–542.

[5] —, Representations and representative functions of Lie groups, III, Ann. of Math. 70 (1959), 85–100.

[6] —, On the algebra of representative functions of an analytic group, Amer. J. of Math. 83 (1961), 111–136.

[7]  —, Pro-affine algebraic groups, Amer. J. of Math. 91 (1969), 1127–1140.

284 Lee

[8] Lee, D. H, and Ta-Sun Wu, On observable subgroups of complex analytic groups, J. of Algebra 173 (1995), 166–179.

Department of Mathematics Case Western Reserve University Cleveland, Ohio 44106, USA dhl@po.cwru.edu

Received March 28, 1998 and in final form May 8, 1998