Journal of Lie Theory Volume 12 (2002) 495–502 C ◯2002 Heldermann Verlag

On Observable Subgroups of Complex Analytic Groups

and

Algebraic Structures on Analytic Homogenous Spaces

Nazih Nahlus

Communicated by K. H. Hofmann

Abstract. Let L be a closed analytic subgroup of a faithfully representable complex analytic group G , let R(G) be the algebra of complex analytic representative functions on G , and let G_{0} be the universal algebraic subgroup (or algebraic kernel) of G .

In this paper, we show many characterizations of the property that

the homogenous space G/L is (representationally) separable, i.e, R(G)^{L} separates the points of G/L. For example, G/L is separable if and only if G_{0} ∩ L is an algebraic subgroup of G_{0} which is (rationally) observable in G_{0} . These characterizations yield new characterizations for the analytic observability of L in G and new characterizations for the existence of a quasi-affine structure on G/L. For example, L is (analytically) observable in G if and only if G/L is separable and L_{0} = G_{0} ∩ L.

Similarly, we discuss a weaker separability of G/L and the exis-

tence of a representative algebraic structure on G/L.

2000 Mathematics Subject Classification: 22E10, 22E45, 22F30, 20G20, 14L15.

Let L be a closed analytic subgroup of a faithfully representable complex analytic group G. Then L is called (analytically) observable in G if every finitedimensional complex analytic representation of L is extendable to a finitedimensional analytic representation of G; or more precisely, if every finitedimensional analytic L-module is a sub L-module of a finite-dimensional analytic G-module [7, p. 166]. Similarly, we have the notion of (rational) observability for algebraic subgroups of linear algebraic groups [1]. If there is no ambiguity, we shall simply use the term ”observable” .

The homogenous space G/L will be called (representationally) separable if R(G)^{L} separates the points of G/L. The homogenous space G/L is said to have a quasi-affine structure if G/L has the structure of a quasi-affine algebraic variety which is compatible with G and R(G) [2, p. 813] (see Definition 1 below). Moreover, G/L is said to have a (representative) algebraic structure if G/L has

ISSN 0949–5932 / $2.50 ^{_{C} ◯} Heldermann Verlag the structure of an algebraic variety which is compatible with G and [R(G)] [3, p. 852] (see Definition 2 below).

The question of observability of L in G has been studied by Lee and Wu in [7]. But there was no explicit mention of the separability of G/L (see Theorem A below). So our Theorem 1 and Corollary 2 completely clarify the role of separability in this respect. The question of having a quasi-affine algebraic structure on G/L has been studied by Hochschild and Mostow [2] where the separability of G/L plays an essential role (see Theorem C below). Moreover, the question of having a (representative) algebraic structure on G/L has been also studied by Hochschild and Mostow [3] where the weaker separability by [R(G)]^{L} plays an essential role (see Theorem D below). So Theorems 1 and 4 and their corollaries clarify the role of separability and weaker separability of G/L which are essential for the existence of such structures on G/L. In addition, Remark 1 clarifies the definition of a quasi-affine structure on G/L. If G is an algebraic group and L is an algebraic subgroup of G, Remark 2 shows that if L is analytically observable in G, then L is rationally observable in G. Finally we use the proof of Theorem 1 to give a new proof, independently of [7], of the ”if part” or the extension part in Thm. 4.5 of [7] stated in Theorem A below.

The proofs in this paper rely heavily on [2], [3], and [7] as well as the recent work by Magid and the author in [8] concerning observable subgroups of pro-affine algebraic groups. We shall assume that the reader is familiar with the basic theory of pro-affine algebraic groups found in [4, Section 2] as well as the basic facts about observable subgroups of linear algebraic groups found in [1] or [6, Thm. 2.1].

Notation and Conventions. Let C be the field of complex numbers, let R(G) be the Hopf algebra of complex analytic representative functions on G, and let G^{∗} be the pro-affine algebraic group associated with R(G). Since G is assumed to be faithfully representable, we shall identify G with its canonical image in G^{∗}. So R(G) = C[G^{∗}] where C[G^{∗}] is the Hopf algebra of polynomial functions on G^{∗}. If A is a subgroup of G^{∗}, let R(G)^{A} be the A-fixed part of R(G) under the translation action a.f(x) = f(xa), and let ^{A}R(G) be the A-fixed part of R(G) under the translation action f.a(x) = f(ax). Similarly, we define [R(G)]^{A} and ^{A}[R(G)], where [R(G)] is the field of fractions of the integral domain R(G). Note that Hochschild and Mostow in [2] and [3], worked with the right cosets L \ G rather than the left cosets G/L and worked with (R(G)^{L})′ = ^{L}R(G) rather than R(G)^{L} where f′(x) = f(x^{−1}).

We recall that the universal algebraic subgroup G_{0} of G may be defined as the subgroup generated by [G, G] and all reductive analytic subgroup of G [9, p. 623]. In fact, G_{0} is the unique maximal normal subgroup of G that is algebraic under all finite-dimensional analytic representations of G. Moreover, G_{0} has a unique irreducible algebraic group structure which is compatible with its analytic group structure [9]. In [7], G_{0} is referred to as the algebraic kernel of G.

For the convenience of the reader, we recall the following (slightly reworded) definitions and results.

Definition 1. [2, top p. 813] A quasi-affine structure for L\G is the structure of a quasi-affine algebraic variety on L\G satisfying the following two conditions:

(1) The variety L \ G is G-homogenous [2, top. 809] in the sense that it satisfies the following conditions.

(a) For each element x of G, the translation action of x on L \ G is an automorphism of the algebraic variety L \ G. (Moreover, G acts transitively on L \ G.)

(b) If P (L \ G) is the algebra of polynomial functions on L \ G, then P (L\G) ⊂ ^{L}R(G) (where the elements of ^{L}R(G) are viewed as functions on L \ G).

(2) The variety L \ G is a G-variety [2, p. 810] in the sense that the translation action of G on L \ G staisfies (a) above, and

(c) for every polynomial f on L \ G and every point v of L \ G, the map f_{v} : G → C defined by f_{v}(x) = f(v.x) is a holomorphic function on G.

Remark 1. The above two conditions (1) and (2) on L \ G are equivalent.

Proof. In fact, if L \ G is a G-variety, then L \ G is G-homogenous by [2, Thm. 1.3]. Converseley, suppose that L \ G is G-homogenous. To prove condition (c) above, Let f be a polynomial function on L \ G, let v = Lg be an element of L\G, let g^{∗} be the translation action of g on L\G, let π : G → L\G be the canonical projection, and let π^{t} : P (L \ G) → ^{L}R(G) be its transpose in view of (b) above. Then f_{v}(x) = f(v.x) = f(Lg.x) = f(g^{∗}(Lgxg^{−1})) = f(g^{∗}(π(gxg^{−1}))) = (π^{t}(f ∘ g^{∗}))(gxg^{−1}). Hence f_{v} is a holomorphic function on G because, g^{∗} is rational by (a), and π^{t}(P (L \ G)) ⊂ ^{L}R(G) by (b), so we have condition (c). Hence (1) and (2) are equivalent.

Definition 2. [3, p. 852] A representative algebraic structure for L \ G is the structure of an irreducible algebraic variety on L \ G satisfying the following two conditions:

(a) For each element x of G, the translation action of x on L \ G is an automorphism of the algebraic variety L \ G.

(b) If F (L\G) is the algebra of rational functions on L\G, then F (L\G) ⊂ L[R(G)] (where the elements of ^{L}[R(G)] are viewed as functions on L \ G).

Theorem A. [7, Thm. 4.5] L is observable in G if and only if L satisfies the following conditions:

(1) L_{0} = L ∩ G_{0} ;

(2) L_{0} is observable in G_{0} (in the categroy of algebraic groups).

Theorem B. [7, Lemmas 4.1, 4.2] If L ∩ G_{0} is an algebraic subgroup of G_{0} , then L^{∗} ∩ G = L (where L^{∗} is defined as in Theorem 1 below). In particular, L^{∗} ∩ G_{0} = L ∩ G_{0} .

Theorem C. [2, Thm. 7.1] G/L has a quasi-affine structure if and only if L satisfies the following conditions:

(1) R(G)^{L} separates the points of G/L;

(2) S(L)α(G) = α(G)^{∗} (See [2] for terminology);

(3) N(L)/L has only a finite number of connected components where N(L) is the normalizer of L in G.

Theorem D. [3, Thm. 3.3] G/L has a representative algebraic structure if and only if L satisfies the following:

(1) [R(G)]^{L} separates the points of G/L;

(2) S(L)α(G) = α(G)^{∗};

(3) N(L)/L has only a finite number of connected components where N(L) is the normalizer of L in G.

Theorem 1. Let L be a closed analytic subgroup of a faithfully representable complex analytic group G, let R(G) be the algebra of complex analytic representative functions on G, let G^{∗} be the pro-affine algebraic group associated with R(G), and let L^{∗} be the algebraic closure of L in G^{∗}. Let G_{0} and L_{0} be the universal algebraic subgroups of G and L respectively. Then the following are equivalent.

(1) G/L is (representationally) separabe, i.e, R(G)^{L} separates the points of G/L.

(2) L^{∗} is observable in G^{∗} (in the category of pro-affine algebraic groups) and L ∩ G_{0} is an algebraic subgroup of G_{0} .

(3) L ∩ G_{0} (and hence L_{0} ) is an observable algebraic subgroup of G_{0} (in the category of algebraic groups).

(4) [R(G)^{L}] = [R(G)]^{L} and L ∩ G_{0} is an algebraic subgroup of G_{0} .

Proof. We shall need the fact that [G, G] and G_{0} are normal in G^{∗} since [G, G] = [G^{∗}, G^{∗}] [5, p. 1149 (last paragraph)]. Suppose R(G)^{L} separates the points of G/L. Let X be the subgroup consisting of all elements of G^{∗} that leave the elements of R(G)^{L} fixed (under the right translation action of G^{∗} on R(G) = C[G^{∗}]) Then X is an algebraic sugroup of G^{∗} and X ∩ G = L since R(G)^{L} separates the points of G/L. Hence L ∩ G_{0} = X ∩ G ∩ G_{0} = X ∩ G_{0} , so L ∩ G_{0} is an algebraic subgroup of G_{0} . Since R(G)^{L} = R(G)^{L^{∗}}, R(G)^{L^{∗}} separates the points of G/L, so in particular, R(G)^{L^{∗}} separates the points of G/L^{∗} ∩ G ≅ G.L^{∗}/L^{∗}. Hence R(G)^{L^{∗}} separates the points of [G, G].L^{∗}/L^{∗}. So if Y = [G, G].L^{∗}, then C[Y]^{L^{∗}} separates the points of Y/L^{∗}. Hence L^{∗} is observable in Y [8, Thm. 3]. Moreover, Y = [G, G].L^{∗} is observable in G^{∗} for being a normal algebraic subgroup of G^{∗} [8, Thm. 1]. Hence L^{∗} is observable in G^{∗}, so (1) implies (2). Suppose (2) holds. Then L^{∗}∩G_{0} is observable in L^{∗} for being a normal algebraic subgroup [8, Thm. 1]. But L^{∗} ∩ G_{0} = L ∩ G_{0} by Theorem B. Hence L ∩ G_{0} is observable in L^{∗} . Consequently, by transitivity, L ∩ G_{0} is also observable in G^{∗} since L^{∗} is given to be observable in G^{∗}, so (2) implies (3).

Now we show (3) implies (2). Since L^{∗} ∩ G_{0} = L ∩ G_{0} by Theorem B, L^{∗} ∩ G_{0} is observable in G_{0} . Although it can be justified using [8, section 2] that that the pro-variety G_{0}.L^{∗}/L^{∗} is isomorphic to G_{0}/G_{0} ∩ L^{∗} which is quasi-affine, to deduce that L^{∗} is observable in G_{0}.L^{∗}, it is simpler to work with the characterization concerning separation of points. Put Z = L^{∗} ∩ G_{0} , so Z is observable in G_{0} . Then C[G_{0}]^{Z} separates the points of G_{0}/Z. If f ∈ C[G_{0}]^{Z}, define f^{+} ∈ C[G_{0}.L^{∗}]^{L^{∗}} by f^{+}(ab) = f(a) if a ∈ G_{0} and b ∈ L^{∗}. In fact, f^{+} is well-defined since f is invariant under Z = L^{∗} ∩ G_{0} . Consequently, C[G_{0}.L^{∗}]^{L^{∗}} separates the points of G_{0}.L^{∗}/L^{∗}. Hence L^{∗} is observable in G_{0}.L^{∗} [8, Thm. 3]. But this last is observable in G^{∗} for being normal in G^{∗} [8, Thm. 1]. Hence L^{∗} is observable in G^{∗} by transitivity, so (3) implies (2).

Now we show that (2) implies (1). Since L^{∗} is observable in G^{∗}, and C[G^{∗}] = R(G), R(G)^{L^{∗}} separates the points of G^{∗}/L^{∗} [8, Thm. 1]. Since R(G)^{L^{∗}} = R(G)^{L}, it follows that R(G)^{L} separates the points of G.L^{∗}/L^{∗} ≅ G/L^{∗} ∩ G. But L^{∗} ∩ G = L by Theorem B. Hence R(G)^{L} separates the points of G/L, so (2) implies (1). Thus (1)-(2)-(3) are equivalent.

Finally, suppose (2) holds. Since L^{∗} is observable in G^{∗} and C[G^{∗}] = R(G), [R(G)^{L^{∗}}] = [R(G)]^{L^{∗}} ([8, Thm. 1] or [7, Thm. 2.2]) But [R(G)^{L^{∗}}] = [R(G)^{L}] and [R(G)]^{L^{∗}} = [R(G)]^{L} since L^{∗} is the algebraic closure of L in G^{∗}. Hence [R(G)^{L}] = [R(G)]^{L}, so (2) implies (4). Converseley, suppose (4) holds. Then [R(G)^{L}] = [C[G^{∗}]]^{L^{∗}}. But this last separates the points of G^{∗}/L^{∗} [8, Prop. 1]. Hence R(G)^{L} separates the points of G^{∗}/L^{∗}. Consequently, R(G)^{L} separates the points of G.L^{∗}/L^{∗} ≅ G/G ∩ L^{∗} = G/L by Theorem B above, so (4) implies (1). This proves Theorem 1.

In view of Theorems A and C, we have the following corollaries.

Corollary 2. The following are equivalent.

(1) L is observable in G.

(2) G/L is separable and L ∩ G_{0} = L_{0} .

(3) L_{0} is observable in G_{0} (in the category of algebraic groups) and L∩G_{0} = L_{0} .

(4) [R(G)^{L}] = [R(G)]^{L} and L ∩ G_{0} = L_{0} .

Corollary 3. G/L has a quasi-affine structure if and only if L satisfies the following conditions:

(1) R(G)^{L} separates the points of G/L, or equivalently, L ∩ G_{0} is an observable algebraic subgroup of G_{0} (in the category of algebraic groups);

(2) S(L)α(G) = α(G)^{∗};

(3) N(L)/L has only a finite number of connected components (where N(L) is the normalizer of L in G).

Theorem 4. The following are equivalent.

(1) [R(G)]^{L} separates the points of G/L.

(2) L ∩ G_{0} is an algebraic subgroup of G_{0} . Proof. Let X be the fixer of [R(G)]^{L} in G^{∗}. Then X is an algebraic subgroup of G^{∗}. If [R(G)]^{L} separates the points of G/L, then X ∩ G = L. Consequently, L∩G_{0} = X ∩G_{0} . Hence L∩G_{0} is an algebraic subgroup of G_{0} , so (1) implies (2). Converseley, suppose L ∩ G_{0} is an algebraic subgroup of G_{0} . Then L^{∗} ∩ G = L by Theorem B above. Since [R(G)]^{L} = [R(G)]^{L^{∗}} and this last separates the points of G^{∗}/L^{∗} [7, Prop. 1], it follows that [R(G)]^{L} separates the points of G.L^{∗}/L^{∗} ≅ G/L^{∗} ∩ G = G/L. This proves Theorem 4.

In view of Theorem D above, we have the following.

Corollary 5. G/L has a representative algebraic structure if and only if L satisfies the following:

(1) [R(G)]^{L} separates the points of G/L, or equivalently, L ∩ G_{0} is an algebraic subgroup of G_{0} .

(2) S(L)α(G) = α(G)^{∗} ;

(3) N(L)/L has only a finite number of connected components (where N(L) is the normalizer of L in G).

Example 1. Let V = C×C where C is the additive group of complex numbers, let G = V.T be the semi-direct product group where T = C acts on V by t(x, y) = (e^{t}x, y) for every t ∈ T, and let L be the diagonal subgroup in V . Then L_{0} = (0) = L ∩ G_{0} . Hence L is observable in G by Theorem A. However, N(L) = V.T^{∗} where T^{∗} consists of the elements t ∈ T such that e^{t} = 1, i.e, of the integral multiples of 2πi, so N(L)/L is an infinite group. Hence, by Theorem B and Theorem C, G/L does not have any quasi-affine structures or even representative algebraic structures although L is observable in G.

Example 2. Let G = SL(2, C) and let L be the unipotent subgroup of

elements of the form ( \begin{matrix}1 & a \\ 0 & 1 \end{matrix} ) where a ∈ C. Then L_{0} = (1) while L ∩ G_{0} = L,

so L is not observable in G (in the category of analytic groups) by Theorem A. However, L is a unipotent algebraic subgroup of G, so L is observable in G (in the category of algebraic groups). Hence G/L is a quasi-affine algebraic variety. Thus G/L has a quasi-affine structure although L is not observable in G (in the category of analytic groups).

Example 3. Let G = C^{∗} × SL(2, C) and let L be the subgroup of elements

of the form (e^{a}, ( \begin{matrix}1 & a \\ 0 & 1 \end{matrix} )) where a ∈ C. Then G_{0} = G, so L ∩ G_{0} is L which is

not an algebraic subgroup of G_{0} . Hence G/L is neither separable by Theorem 1, nor weakly separable by Theorem 4. Moreover, L is not observable in G by Corollary 2.

Remark 2. Supoose that G is an algebraic group and L is an algebraic subgroup of G. If L is analytically observable in G, then L is rationally observable in G. However, the converse is false in general.

Proof. The converse is false in general by Example 2. So suppose that L is analytically observable in G. Then L_{0} is rationally observable in G_{0} by Theorem A. Moreover, G_{0} is rationally observable in G for being normal in G. Hence L_{0} is rationally observable in G. Now L/L_{0} is a unipotent algebraic group since L = L_{u} . P and L_{0} = N . P for every maximal reductive algebraic subgroup P of L where L_{u} is the unipotent radical of L and N is the radical of [L, L]. Hence every multiplicative rational character on L/L_{0} is trivial. Since L_{0} is rationally observable in G, it follows that L is rationally observable in G [10, Cor. 2 (2)].

Finally, we use the proof of Theorem 1 to give a new proof independently of [7] of the ”if part” or the extension part in Thm. 4.5 of [7].

Theorem A (if part). L is observable in G if L satisfies the following conditions:

(1) L_{0} = L ∩ G_{0} ;

(2) L_{0} is observable in G_{0} (in the categroy of algebraic groups).

Proof. Let L^{+} be the pro-affine algebraic group associated with R(L), so R(L) = C[L^{+}]. Then the restriction map R(G) → R(L) yields a canonical map

f : L^{+} → G^{∗}

and note that f(L^{+}) = L^{∗}. Now Let V be a finite-dimensional analytic representation of L. Then V is rational representation of L^{+} . To obtain the extension to G, first we show that it suffices to have that f is injective. If this is the case, then V becomes a rational representation of L^{∗}. Our two given assumptions imply that L ∩ G_{0} is an observable algebraic subgroup of G_{0} (in the category of algebraic groups). Hence L^{∗} is observable in G^{∗} by the implication (3) ⇒ (2) in Theorem 1. (Note that the proof of this implication does not rely on Theorem B, so it is independent of [7]). Hence V can be extended to a finite-dimensional rational representation of G^{∗} whose restriction to G yields the required extension.

Now we show that f is indeed injective by showing that the induced map f^{+} : L^{+}/L ∩ G_{0} → G/G_{0} is injective. The inclusion map of L into G yields an injection of abelian analytic groups i : L/L ∩ G_{0} → G/G_{0} . So every representative function on L/L ∩ G_{0} can be extended to a representative function on G/G_{0} . Hence the canonical map R(G/G_{0}) → R(L/L ∩ G_{0}) is surjective. But R(G/G_{0}) = R(G)^{G_{0}} = C[G^{∗}]^{G_{0}} = C[G^{∗}/G_{0}]. Similarly R(L/L ∩ G_{0}) = C[L^{+}/L ∩ G_{0}]. Hence the canonical map (induced by f) C[G^{∗}/G_{0}] → C[L^{+}/L ∩ G_{0}] is surjective. So the canonical map f^{+} : L^{+}/L ∩ G_{0} → G/G_{0} is injective. Hence Ker(f) ⊂ L∩G_{0} . But L∩G_{0} = L_{0} and f is injective on L. Hence Ker(f) is trivial, and our proof is complete.

References

[1] Bialynicki-Birula, G. Hochschild, and G. D. Mostow, Extensions of representations of algebraic groups, Amer. J. Math. 85 (1963), 131–144.

[2] Hochschild, G., and G. D. Mostow, Affine embeddings of complex analytic homogenous spaces, Amer. J. Math. 87 (1965), 807–839.

[3] —, Representative algebraic structures on complex analytic homogenous spaces, Amer. J. Math. 88 (1966), 847–866.

[4]  —, Pro-affine algebraic groups, Amer. J. Math. 91 (1969), 1127–1140.

[5] —, Complex analytic groups and Hopf algebras, Amer. J. Math. 91 (1969), 1141–1151.

[6] Grosshans, F. D., “Algebraic homogenous spaces and invariant theory,” Lecture Notes in Mathematics 1673, Springer-Verlag, Berlin, 1997, vi+148 pp.

[7] Lee, Dong Hoon, and Wu, Ta-Sun, On observable subgroups of complex analytic groups, J. Algebra 173 (1995), 166–179.

[8] Magid, A. R., and Nahlus, N., Provarieties and observable subgroups of pro-affine algebraic groups, Journal of Group Theory, (to appear).

[9] Nahlus, N., Representative functions on complex analytic groups, Amer. J. Math. 116 (1994), 621–636.

[10] —, Note on observable subgroups of linear algebraic groups and a theorem of Chevalley, Journal of Lie Theory 12 (2002), 301–304.

Nazih Nahlus Department of Mathematics American University of Beirut Beirut, Lebanon nahlus@aub.edu.lb

Received May 29 , 2001 and in final form July 9, 2001

Editorial Remark. Through an editorial oversight, the Communicating Editor of the author’s paper [10] was stated incorrectly. That article was communicated by Martin Moskowitz.