Journal of Lie Theory Volume 9 (1999) 39–38 ◯^{C}1999 Heldermann Verlag

On the image of a generalized d-plane transform on ℝ^{n}

Elevtherios Symeonidis

Communicated by J. Hilgert

Abstract. The generalized d-plane transform of a function f on ℝ^{n} is defined on a set E of d-dimensional affine subspaces (“d-planes”) of ℝ^{n} by integration of f over each subspace in E. In general, it renders less information about the unknown function f than in the special case of the well known Radon d-plane transform, where E contains every d-plane in ℝ^{n}. We study the case where E appears as an orbit of a matrix group and characterize the range of the spaces of Schwartz functions and of smooth ones with compact support.

Introduction

The Radon d-plane transform of a function f on ℝ^{n} is a function on the set of d-dimensional affine subspaces (“d-planes”) of ℝ^{n}, obtained by integration of f over each such subspace. For d = n − 1, the case of hyperplanes, it was first introduced and studied by Radon in his famous work of 1917 [12]. For d < n − 1, the idea goes back to John, who considered line integrals of point functions in ℝ^{3} in connection with the ultrahyperbolic differential equation [10]. However, the first one who systematically studied this transform was Helgason. Beside inversion formulas, he investigated the range of certain function spaces, such as C_{c}^{∞}(ℝ^{n}), the smooth functions with compact support, or S(ℝ^{n}), the Schwartz functions (see [9]). The range of C_{c}^{∞}(ℝ^{n}) under the Radon d-plane transform has been described by Helgason in terms of the so called “moment conditions”. For d = n − 1 (the Radon transform), these characterize the range of S(ℝ^{n}), too [9]. However, this is no longer true for d < n − 1, as Gonzalez later showed [6].

In the already mentioned article of 1938 John showed that the line functions in ℝ^{3}, which are obtained by integration of point functions over straight lines, satisfy certain differential equations, which characterize this class of line functions [10]. The idea was further developed by Gel’fand, Gindikin and Graev in 1980, who gave a characterization of the range of S(ℝ^{n}) in terms of a system of second order partial differential equations but without complete proof [3]. In 1984 Grinberg

ISSN 0949–5932 / $2.50 ◯^{C} Heldermann Verlag used both moment conditions and the differential operators of Gel’fand et al. to determine the range [7]. The proof of the assertion of Gel’fand et al. was given in detail by Richter in 1986 [13]. It was his great achievement to observe that the differential operators arise from infinitesimal operators of the group of euclidean motions in ℝ^{n} [14]. Later, Gonzalez and Kurusa gave other proofs, the former in the group theoretic frame of Richter, the latter following John’s method of parametrizing the d-planes [11], [6].

In this article we study the generalized situation, where the d-plane transform is only defined on certain subsets E of the set G(d, n) of all d-planes in ℝ^{n}. In practice, particularly in computer tomography, where the theory applies decisively, measurements can only be made for a finite set of d-planes, so it is of natural interest to consider the whole problem in the restriction to subclasses of d-planes. It is of great practical importance to see how the main facts such as inversion formulae and range theorems depend on the geometry of E. Gel’fand, Graev and others have studied the complex case, where E is an n-dimensional analytic submanifold of the set of all (complex) d-planes in ℂ^{n}, and have termed E “admissible”, if it is possible to recapture a function from its integrals over the planes in E (see for instance [4]). Felix has treated some more general settings [2]. In this work we show that the group theoretic approach, as it has been introduced by Helgason and applied by Richter and Gonzalez with great success to the range problem, bears its fruits in a greater generality of cases, provided that E is an orbit of a matrix group. So we restrict ourselves to such cases. Moreover, the group theoretic treatment of the generalized d-plane transform reveals the meaning of the otherwise mysterious construction of the differential operators which characterize the range in [14] (here: remark 2 to lemma 5.2).

In the sequel, by “classical case” we shall mean the case E = G(d, n). The Radon d-plane transform, which refers to it, has been extensively studied through the past decades by Helgason, Richter and Gonzalez, as already mentioned.

The organization of the paper is as follows. The set E is introduced in section 1. In section 2 the notion of rapidly decreasing functions on E and the Fourier transform are defined. Section 3 deals with the group theoretic nature of the differential operators which serve to characterize the range. Here we follow Richter [14] and correct the way in which he defines these operators. This part can be skipped during the first reading. The d-plane transform, the moment conditions and the differential operators in connection to it are introduced in section 4. In 5, the range of S(V), the set of Schwartz functions on the vector space V , is characterized by both moment conditions and differential operators. The main work is done here. Section 6 covers a lot of special cases (the classical case included), where the differential operators alone characterize the range of S(V). In section 7 it turns out that for d = n − 1 the only example that fits into our framework is the classical case itself. The range of C_{c}^{∞}(V) is determined with the help of a “polar coordinate” version of the classical Paley-Wiener theorem, due to Helgason [8]. This is done in section 8. Finally, the theory is applied to some concrete situations (section 9). The precise characterization of those cases, where we can dispense with the moment conditions, is still an open problem.

The author would like to stress that the main facts concerning the range of C_{c}^{∞}(ℝ^{n}) and S(ℝ^{n}) in the classical case are here recovered as special cases. In this sense, the classical results are truly generalized.

1. Fundamental concepts

Let H be a Lie group, ρ a (C^{∞}-) representation of H on a finite-dimensional vector space V . The representation ρ gives rise to a semi-direct product H ×_{s} V , where multiplication is given by

(h_{1}, v_{1}) · (h_{2}, v_{2}) := (h_{1}h_{2}, v_{1} + ρ(h_{1})v_{2})

(from now on we write h · v instead of ρ(h)v).

The Lie group H ×_{s} V operates transitively on V :

(h, v) · v_{0} := v + h · v_{0} .

It also operates transitively on G(d, V), the set of d-dimensional affine subspaces of V (1 ≤ d ≤ n − 1, n = dim V ) :

(h, v) · E := v + h · E (h · E = {h · z|z ∈ E}) .

We fix a d-dimensional linear subspace E of V and denote by H_{E} the isotropy subgroup of E in H. Then H_{E} ×_{s} E is the isotropy subgroup of E in H ×_{s} V . The orbit of E will be identified with ^{H×_{s}V} /_{H_{E}×_{s}E} and will be given the differentiable structure of this homogeneous space. ^{H×_{s}V} /_{H_{E}×_{s}E} is in fact a vector bundle over ^{H}/_{H_{E}}: If σ : W → H is a local cross section of an open set W ⊆ ^{H}/_{H_{E}} into H and U a complement of E in V , we obtain a local trivialization by the mapping (ḣ, u) ↦ (σ(ḣ), σ(ḣ) · u)(H_{E} ×_{s} E) (ḣ = hH_{E}) of W × U into H×_{s}V /_{H_{E}×_{s}E}. We put E := ^{H×_{s}V} /_{H_{E}×_{s}E} and (ḣ, x) for (h, x)(H_{E} ×_{s} E).

The contragredient representation ρ^{∗},

[ρ^{∗}(h)ξ](v) := ξ[ρ(h)^{−1}v] (v ∈ V, ξ ∈ V^{∗}) ,

leads analogously to a semi-direct product H ×_{s} V^{∗}. We write h^{∗} · ξ instead of ρ^{∗}(h)ξ. If E^{⊥} ⊆ V^{∗} denotes the orthogonal subspace to E, then the set

E^{∗} := {(ḣ, ξ)|ξ ∈ h^{∗} · E^{⊥}}

can be given the structure of a vector bundle over ^{H}/_{H_{E}} via the mappings (ḣ, w) ↦ (ḣ, σ(ḣ)^{∗} · w) of W × E^{⊥} into E^{∗} (notation as above). (Note that h^{∗} · E^{⊥} does not depend on h ∈ hH_{E}.) The set E^{∗} will be called “dual bundle” (its fibres can be naturally identified with the dual spaces of the fibres of E).

2. Rapidly decreasing functions on E and the Fourier transform

Gonzalez introduced a class of functions on G(d, ℝ^{n}), the so called rapidly decreasing functions [6]. They can be defined on E in a similar way. By a local cross section of a compact set M ⊆ ^{H}/_{H_{E}} into H we shall always mean a local C^{∞}- cross section of an open neighborhood of M into H. Let U be a complement of E in V and introduce linear coordinates (u_{1}, …, u_{n−d}) on U. The vector bundle E is of fibre type U, as already shown.

Definition 2.1. A C^{∞}-function φ on E is called rapidly decreasing if for all multiindices α, β ∈ ℤ_{+}^{n−d}, all differential operators T on ^{H}/_{H_{E}} and all compact sets M ⊆ ^{H}/_{H_{E}} admitting a local cross section σ into H

sup_{ḣ∈M, u∈U} |u^{α}Tḣ \frac{∂^{β}}{∂u^{β}} φ(ḣ, σ(ḣ) · u)| < ∞ .

(We have put u^{α} := ∏_{i=1}^{n−d} u_{i}^{α_{i}}, \frac{∂^{β}}{∂u^{β}} := ∏_{i=1}^{n−d} \frac{∂^{β_{i}}}{∂u_{i}^{β_{i}}} .)

The above definition does not depend on the choice of U or of its basis. Rapidly decreasing functions on E^{∗} can be defined similarly (E^{⊥} replaces U). We write S(E), S(E^{∗}) for these classes of functions.

Let f ∈ C^{∞}(V^{∗}) be a function of the Schwartz class. We define a function φ_{f} on E^{∗} by

φ_{f}(ḣ, ξ) := f(ξ) .

Lemma 2.2. φ_{f} ∈ S(E^{∗}).

Proof. φ_{f} ∈ C^{∞}(E^{∗}) by the local trivializations of the vector bundle E^{∗}. We keep the notation of definition 2.1 and consider a covering of M by open, relatively compact subsets O_{1}, …, O_{k} having the additional property that each \overline{O_{i}} is contained in a chart of ^{H}/_{H_{E}} and that there exists a local cross section on the union of all \overline{O_{i}}. Under these assumptions it suffices to verify the property of definition 2.1 for each O_{i} separately. In what follows, \overline{O_{i}} will be regarded as a compact set in an ℝ^{m}.

Let ∥·∥_{V^{∗}} be a norm on V^{∗}, ∥·∥ the corresponding norm of linear operators on the space of endomorphisms of V^{∗}. If T is a differential operator of order r on H/_{H_{E}}, let

C := max{ sup_{0≤|s|≤r} sup_{ḣ∈O_{i}} ∥∂^{s}ρ^{∗}(σ(ḣ))∥, sup_{ḣ∈O_{i}} ∥ρ^{∗}(σ(ḣ))^{−1}∥} < ∞

(s = (s_{1}, …, s_{m}) ∈ ℤ_{+}^{m} is a multiindex, |s| := ∑_{i=1}^{m} s_{i}, and ∂^{s} is defined as in definition 2.1 (O_{i} ⊆ ℝ^{m})). Let (w_{1}, …, w_{n−d}) denote the vector of coordinates with respect to a basis in E^{⊥}. The expression

|w^{α}Tḣ(\frac{∂}{∂w_{1}} , …, \frac{∂}{∂w_{n−d}})^{β}φ_{f}(ḣ, σ(ḣ)^{∗} · w)|

= |( (σ(ḣ)^{−1})^{∗} · (σ(ḣ)^{∗} · w) )^{α}(\frac{∂}{∂w_{1}} , …, \frac{∂}{∂w_{n−d}})^{β}Tḣf(σ(ḣ)^{∗} · w)|

can then be estimated by a linear combination of terms of the form

C^{l}∥σ(ḣ)^{∗} · w∥_{V^{∗}}^{l′} · |Df(σ(ḣ)^{∗} · w)|

(D a constant coefficient differential operator, l, l′ ∈ ℤ_{+}). However, since f ∈ S(V^{∗}), the latter are bounded for ḣ ∈ O_{i}, w ∈ E^{⊥}.

Let p_{U} denote the projection of V onto U, according to the decomposition V = U ⊕ E. In order to define a Fourier transform on E, we have to assume that a translation-invariant integral du on U is left invariant by the mappings

u ⟼ p_{U}(s · u) , s ∈ H_{E} .

Definition 2.3. The Fourier transform φ̃ of a rapidly decreasing function φ on E is a function on the dual bundle E^{∗}, defined by

φ̃(ḣ, ξ) := ∫_{U} φ(ḣ, h · u)e^{−i⟨h·u,ξ⟩}du , ξ ∈ h^{∗} · E^{⊥}, ⟨h · u, ξ⟩ := ξ(h · u) .

(The right hand side does not depend on h ∈ hH_{E} because of the assumption above.)

In fact, the Fourier transform on E is the classical Fourier transform on its fibres. It therefore can be extended to functions which are integrable on the fibres of E.

Proposition 2.4. φ ∈ S(E) ⇒ φ̃ ∈ S(E^{∗}). The proof is straightforward and is left to the reader.

3. The left regular representation of H ×_{s} V on C^{∞}(V) and the kernel of its differential

As we have seen in the first section, the Lie group H ×_{s} V operates transitively on V . The subgroup H (identified with H ×_{s} {0}) is the isotropy subgroup of 0 ∈ V , so V can be identified (set-theoretically and as a manifold) with ^{H×_{s}V} /_{H}. The left regular representation λ of H ×_{s} V on C^{∞}(V) is defined by

[λ(h, v)f](v′) = f[(h, v)^{−1} · v′] = f(h^{−1} · (v′ − v)) (f ∈ C^{∞}(V) ) .

It follows that the subspace of Schwartz functions S(V) is λ-invariant. The representation λ can be differentiated on C^{∞}(V). For Y ∈ 𝔥 ×_{s} V , the Lie algebra of H ×_{s} V , and f ∈ C^{∞}(V) we have

dλ(Y)f(v) = \frac{d}{dt}f(exp(−tY) · v)|_{t=0} .

If f ∈ S(V), the right hand side remains in S(V). This shows that S(V) is dλ-invariant, so λ can be differentiated as a representation on S(V). Since dλ is a representation of 𝔥 ×_{s} V , it can be extended to a representation of the universal enveloping algebra U(𝔥 ×_{s} V).

Let X_{1}, …, X_{m} be a basis of 𝔥, Z_{1}, …, Z_{n} one of V (being identified with its Lie algebra). By the Poincaré-Birkhoff-Witt theorem the elements X_{1}^{i_{1}} … X_{m}^{i_{m}}Z_{1}^{j_{1}} … Z_{n}^{j_{n}}, i_{k}, j_{l} ∈ ℤ_{+}, form a basis of U(𝔥 ×_{s} V).

Definition 3.1. P ∈ U(𝔥 ×_{s} V) is said to have X -degree M ≥ 0 if P can be written as a linear combination of basis vectors X_{1}^{i_{1}} … X_{m}^{i_{m}}Z_{1}^{j_{1}} … Z_{n}^{j_{n}} with i_{1} + … + i_{m} ≤ M, where at least one vector appears with i_{1} + … + i_{m} = M.

This definition does not depend on the (ordered) basis X_{1}, …, X_{m} of 𝔥. It comes from the canonical filtration of U(𝔥). An X -degree is assigned to every nonzero element P ∈ U(𝔥 ×_{s} V).

The adjoint representation Ad of H×_{s}V as well as its differential ad extend to representations on U(𝔥 ×_{s} V).

Lemma 3.2. If P ∈ U(𝔥 ×_{s} V) has X -degree M > 0, then for every Z ∈ V adZ(P) has X -degree < M or is equal to 0.

The proof follows immediately from the relations adZ(X_{k}) ∈ V (1 ≤ k ≤ m) for the derivations adZ.

Let π_{V} : U(𝔥 ×_{s} V) → U(V) be the canonical projection, defined by π_{V} (X_{1}^{i_{1}} … X_{m}^{i_{m}}Z_{1}^{j_{1}} … Z_{n}^{j_{n}}) = 0 for i_{1} + … + i_{m} > 0, π_{V} (Z_{1}^{j_{1}} … Z_{n}^{j_{n}}) = Z_{1}^{j_{1}} … Z_{n}^{j_{n}}. We define a subspace K(𝔥 ×_{s} V) of U(𝔥 ×_{s} V) as follows:^{1}

An element P ∈ U(𝔥×_{s}V) of X -degree M ≥ 0 belongs to K(𝔥×_{s}V) iff
for all j_{1}, …, j_{n} ∈ ℤ_{+}, j_{1}+…+j_{n} ≤ M, π_{V} (adZ_{1})^{j_{1}} … (adZ_{n})^{j_{n}}(P) =
0. Moreover, 0 ∈ K(𝔥 ×_{s} V).

For later purposes we write K^{1}(𝔥×_{s} V) for {P ∈ K(𝔥×_{s} V) | P has X-degree 1}∪ {0}.

Now we are ready to state and prove the main result of this section.

Theorem 3.3. ker(dλ) = K(𝔥 ×_{s} V).

Proof. We first prove the following statement: If an element P with π_{V} (P) = 0 satisfies

dλ(P) = dλ(Ad(exp Z)P)

for every Z ∈ V , then P ∈ ker(dλ).

Let P ≠ 0 be such an element,

P = ∑_{J} c_{J}P_{J}, J = (i_{1}, …, j_{n}), P_{J} = X_{1}^{i_{1}} … X_{m}^{i_{m}}Z_{1}^{j_{1}} … Z_{n}^{j_{n}}, c_{J} ∈ ℂ \ {0} .

For f ∈ C^{∞}(V) we have

dλ(P_{J})(f)(v) = \frac{∂^{|J|}}{∂t^{J}} f(v(t) + h(t) · v)|_{t=0} , where

t = (t_{1}, …, t_{m}, t_{m+1}, …, t_{m+n}), h(t) = exp(−t_{m}X_{m}) … exp(−t_{1}X_{1}) ∈ H ,

v(t) = exp(−t_{m+n}Z_{n}) … exp(−t_{m+1}Z_{1}) = − ∑_{i=1}^{n} t_{m+i}Z_{i} ∈ V ,

1Richter’s definition leads to a much greater space K(n) (following his notation), and the relation ‘K(n) ⊆ Kerλ’ fails to hold in [14, Proposition 3]. For example, if P belongs to K^{M}(n), then P + U^{M−1}(n) ⊆ K(n), but of course P + U^{M−1}(n) ⊈ Kerλ. Furthermore, the use of the operators Z_{i} as Ad(exp(Z_{i}) − Id instead of adZ_{i} in the definition of K^{M}(n) makes it much harder to deduce the last equation in the proof of Proposition 3 if Y is not just one of the Z_{1}, …, Z_{n}, but a linear combination of them.

\frac{∂^{|J|}}{∂t^{J}} = \frac{∂^{i_{1}+...+i_{m}+j_{1}+...+j_{n}}}{∂t_{1}^{i_{1}} … ∂t_{m}^{i_{m}}∂t_{m+1}^{j_{1}} … ∂t_{m+n}^{j_{n}}} .

The relation exp(Ad(g)Y) = g exp(Y)g^{−1}, Y ∈ 𝔥 ×_{s} V , g ∈ H ×_{s} V , implies

dλ(Ad(g)P)(f)(v) = dλ(P)(f ∘ L_{g})(g^{−1} · v) for P ∈ U(𝔥 ×_{s} V) ,  (1)

L_{g} denoting left translation by g. We obtain

dλ(Ad(exp v)P_{J})(f)(v) = dλ(P_{J})(f ∘ L_{v})(0) = \frac{∂^{|J|}}{∂t^{J}} f(v + v(t) + h(t) · 0)|_{t=0} = 0 ,

since \frac{∂^{|J|}}{∂t^{J}} contains at least one derivative with respect to a t_{i}, 1 ≤ i ≤ m. The assumption on P leads to dλ(P)(f)(v) = 0, which holds for all v ∈ V , f ∈ C^{∞}(V). This proves the statement.

Let now P ∈ K(𝔥 ×_{s} V) having X -degree M. For all Z ∈ V and all j_{1}, …, j_{n} ∈ ℤ_{+}, j_{1} + … + j_{n} = M − 1, we obtain by the use of Ad(exp Z) = e^{adZ}:

dλ(Ad(exp Z)(adZ_{1})^{j_{1}} … (adZ_{n})^{j_{n}}P) = dλ((adZ_{1})^{j_{1}} … (adZ_{n})^{j_{n}}P) .

From the above statement it follows that the right hand side is equal to 0. We repeat this argument with j_{1}+…+j_{n} = M −2, M −3 etc. and obtain inductively dλ(P) = 0.

For the converse we observe that ker(dλ) is Ad(H ×_{s} V)-invariant, as equation (1) shows. By differentiation we obtain (adZ)(ker(dλ)) ⊆ ker(dλ) for every Z ∈ V (in fact, Ad(g) and ad(Z) operate on the finite dimensional subspaces of elements of X -degrees ≤ M). Therefore it suffices to show that if P ∈ ker(dλ) it holds π_{V} (P) = 0. Let P_{0} = π_{V} (P). For f ∈ C^{∞}(V), v ∈ V we obtain:

0 = dλ(P)(f)(v) = dλ[Ad(exp v)P](f)(v) = dλ(P)(f ∘ L_{exp v})(0) by (1)
= dλ(P_{0})(f ∘ L_{exp v})(0) + dλ(P − P_{0})(f ∘ L_{exp v})(0) = dλ(P_{0})(f ∘ L_{exp v})(0)
= dλ[Ad(exp v)P_{0}](f)(v) = dλ(P_{0})(f)(v) ,

so dλ(P_{0}) = 0 and P_{0} = 0, since dλ is injective on U(V).

Remark . The definition of the subspace K(𝔥 ×_{s} V) is suggested by Corollary 3.6 in [1].

4. The d-plane transform

We return to the setting of the first two sections. E is a d-dimensional subspace of V (1 ≤ d ≤ n − 1), U a complement of E in V . The following considerations contain the Radon transform (d = n − 1) as a special case.

Let us introduce translation invariant integrals dz and du on E and U respectively. The integral dy = dz du is then translation invariant on V . From now on we assume that H_{E} acts by unimodular automorphisms on E (that is of determinant ±1). This, combined with the condition needed for the Fourier transform on E, leads to the assumption that H_{E} acts unimodularly on both E and V . Moreover, we can assume that H operates by unimodular automorphisms on V ; if this is not the case, we only have to replace ρ(h) by δ(h)^{−1/n}ρ(h), where δ(h) denotes the modulus (= absolute value of the determinant) of ρ(h).

Definition 4.1. Let f be an integrable function on V . The d-plane transform of f is a function f̂ on E, defined by

f̂(ḣ, x) = ∫_{E} f(x + h · z)dz .

The new function is well defined almost everywhere, due to the unimodularity assumption above. The following lemma, often called “Projection-Slice theorem”, is of great importance.

Lemma 4.2. Let f ∈ S(V), f̃ ∈ S(V^{∗}) being the (classical) Fourier transform of f. The d-plane transform f̂ is then integrable on every fibre of E, and we have:

˜f̂(ḣ, ξ) = f̃(ξ) , ξ ∈ h^{∗} · E^{⊥} .

Proof. Clearly, f̂ is everywhere defined and integrable on the fibres by the Fubini theorem. We have:

˜f̂(ḣ, ξ) = ∫_{U} f̂(ḣ, h · u)e^{−i⟨h·u,ξ⟩}du = ∫_{U} ∫_{E} f(h · (u + z))e^{−i⟨h·(u+z),ξ⟩}dz du

(⟨h · z, ξ⟩ = 0)

= ∫_{V} f(h · y)e^{−i⟨h·y,ξ⟩}dy = ∫_{V} f(y)e^{−i⟨y,ξ⟩}dy = f̃(ξ) .

Lemma 4.3. For f ∈ S(V) we have f̂ ∈ S(E).

Proof. Clearly, f̂ ∈ C^{∞}(E). From lemmas 4.2 and 2.2 it follows that ˜ f̂ ∈ S(E^{∗}). The desired relation is now obtained from the analogue of proposition 2.4 for the inverse Fourier transform.

An important property of the d-plane transforms f̂ is that they satisfy the so called “moment conditions”.

Proposition 4.4. Let φ ∈ S(V)^{∧}. For every m ∈ ℤ_{+} there exists a homogeneous polynomial P_{m} of degree m on V^{∗}, such that for every h ∈ ^{H}/_{H_{E}}, ξ ∈ h^{∗} · E^{⊥}:

∫_{U} φ(ḣ, h · u) · ⟨h · u, ξ⟩^{m}du = P_{m}(ξ) .  (2)

Proof. Let φ = f̂, f ∈ S(V). We have:

∫_{U} φ(ḣ, h · u)⟨h · u, ξ⟩^{m}du = ∫_{U} ∫_{E} f(h · (u + z))⟨h · (u + z), ξ⟩^{m}dz du
= ∫_{V} f(h · y)⟨h · y, ξ⟩^{m}dy = ∫_{V} f(y)⟨y, ξ⟩^{m}dy .

Let S_{M}(E) := {φ ∈ S(E)|φ satisfies (2)}. We have shown: S(V)^{∧} ⊆ S_{M}(E).

The functions in S(V)^{∧} have a further common property. They are annihilated by certain differential operators.

Let ν be the left regular representation of H ×_{s}V on C^{∞}(E). For f ∈ S(V) we compute:

[ν(h, v)f̂](ḣ′, x′) = f̂[(h, v)^{−1} · (ḣ′, x′)] = f̂(h^{−1}ḣ′, h^{−1} · x′ − h^{−1} · v)
= ∫_{E} f(h^{−1} · x′ − h^{−1} · v + h^{−1}h′ · z)dz = ∫_{E} f((h, v)^{−1} · (x′ + h′ · z))dz

= ∫_{E}[λ(h, v)f](x′ + h′ · z)dz = [λ(h, v)f]^{∧}(ḣ′, x′)

(λ denoting the left regular representation of H ×_{s} V on C^{∞}(V)), so

[λ(g)f]^{∧} = ν(g)f̂, g ∈ H ×_{s} V ,  (3)

which means that the operator f ↦ f̂ intertwines λ and ν. Differentiation of (3) leads to

[dλ(Y)f]^{∧} = dν(Y)f̂

for Y in 𝔥 ×_{s} V and then in U(𝔥 ×_{s} V). Thus, if Y ∈ ker(dλ), dν(Y)f̂ = 0 for all f ∈ S(V)^{∧}.

Let S_{D}(E) := {φ ∈ S(E)|dν(Y)φ = 0 for Y ∈ ker(dλ)}. We have shown: S(V)^{∧} ⊆ S_{D}(E). Thus, S(V)^{∧} ⊆ S_{D}(E) ∩ S_{M}(E).

5. The image of the d-plane transform

In this section we prove the relation

S(V)^{∧} = S_{D}(E) ∩ S_{M}(E) .  (4)

For this we have to impose the following two conditions on the action of the Lie group H:

(i) The mapping from H × E^{⊥} into V^{∗}, (h, w) ↦ h^{∗} · w, is surjective and its restriction to H × (E^{⊥} \ {0}) a submersion.

(ii) If h^{∗} · w_{1} = w_{2}, w_{1}, w_{2} ∈ E^{⊥}, h ∈ H, then there exists s ∈ H_{E} such that s^{∗} · w_{1} = w_{2}.

Both conditions are satisfied in the classical case (H = SO(n), V = ℝ^{n}).^{2} Lemma 5.1. The d-plane transform is an injective mapping on S(V).

2In the first and last example of section 9 it is shown that they are also necessary for a relation like (4).

The proof follows from lemma 4.2 and the surjectivity in assumption (i) above.

For (4) it remains to be proven that for φ ∈ S_{D}(E) ∩ S_{M}(E) there exists f ∈ S(V) such that φ = f̂. We proceed in three steps:

1. There exists a function F ∈ C^{∞}(V^{∗} \ {0}) satisfying φ̃(ḣ, ξ) = F (ξ).

2. F ∈ C^{∞}(V^{∗}).

3. F ∈ S(V^{∗}).

It then holds for f ∈ S(V) with f̃ = F:

φ̃(ḣ, ξ) = F (ξ) = f̃(ξ) = ˜ f̂(ḣ, ξ)

by lemma 4.2, so φ̃ = ˜ f̂, and by inverse Fourier transform, φ = f̂.

First step

Let X_{1}, …, X_{m} be a basis of 𝔥. For every w ∈ V^{∗} we denote by H_{w} the isotropy subgroup of w in H and by 𝔥_{w} its Lie algebra.

Lemma 5.2. Let l = inf{dim H_{w}|w ∈ V^{∗}\{0}}, O = {w ∈ V^{∗}\{0}| dim H_{w} = l}.

a) O is a dense open subset of V^{∗} \ {0}.

b) Let w_{0} ∈ O. For every w in a dense open neighborhood of w_{0} in O there

exists a basis

Y_{j}(w) = ∑_{k=1}^{m} a_{jk}(w)X_{k} , 1 ≤ j ≤ l ,

of 𝔥_{w}, such that the coefficients a_{jk} are homogeneous polynomials in w ∈ V^{∗}.

Proof. If we differentiate with respect to t the equation ρ^{∗}[exp t(a_{1}X_{1} + … + a_{m}X_{m})]w − w = 0, we obtain:

Y = ∑_{k=1}^{m} a_{k}X_{k} ∈ 𝔥_{w} ⟺ dρ^{∗}(Y)w = 0 ⟺ ∑_{k=1}^{m} a_{k}dρ^{∗}(X_{k})w = 0 .  (5)

a) O ⊆ V^{∗} is the set of vectors w ∈ V^{∗}, where the system (dρ^{∗}(X_{1})w, …, dρ^{∗}(X_{m})w) has maximal rank, that is m − l.

b) Let us assume that (dρ^{∗}(X_{k})w)_{1≤k≤m−l} is a basis of ⟨dρ^{∗}(X_{k})w|1 ≤ k ≤ m⟩ for w in a dense open neighborhood of w_{0} in O. If we bring the terms with m−l < k ≤ m in (5) to the right hand side (and choose a basis on V^{∗}), we obtain for every choice of a_{m−l+1}, …, a_{m} an n × (m − l)-linear system, whose equations are linear in w. The statement of the lemma follows by applying Cramer’s rule to an (m − l) × (m − l)-part of the system and multiplying the vector (a_{1}, …, a_{m}) by the common denominator.

Remark . 1. According to the construction given in the proof, the polynomials a_{jk} have the same degree of homogeneity.

2. If we consider the polynomials a_{jk} as elements of the symmetric (= universal enveloping) algebra of V , then the Y_{j} become the operators in K^{1}(𝔥 ×_{s} V) (see section 3):

P_{j} := ∑_{k=1}^{m} X_{k}a_{jk} ∈ K^{1}(𝔥 ×_{s} V) .

This can be seen as follows (Z_{1}, …, Z_{n} a basis of V ):

0 = ∑_{k=1}^{m} a_{jk}(ξ)dρ^{∗}(X_{k})ξ

implies

0 = ⟨∑_{k=1}^{m} a_{jk}(ξ)dρ^{∗}(X_{k})ξ, Z_{l}⟩ = ∑_{k=1}^{m} a_{jk}(ξ)⟨dρ^{∗}(X_{k})ξ, Z_{l}⟩

= − ∑_{k=1}^{m} a_{jk}(ξ)⟨ξ, dρ(X_{k})Z_{l}⟩ = ∑_{k=1}^{m} a_{jk}(ξ)⟨ξ, [Z_{l}, X_{k}]⟩ = ⟨ξ, (adZ_{l})P_{j}⟩ .

This gives a geometrical meaning to dim K^{1}(𝔥 ×_{s} V) = l. We observe that for φ ∈ S(E), T ∈ 𝔥 ×_{s} V , dν(T)φ remains in S(E). Therefore we can define the Fourier transform of every dν(T), T ∈ 𝔥 ×_{s} V , and then for T ∈ U(𝔥 ×_{s} V), by iteration:

d̃ν(T)φ̃ := [dν(T)φ]^{∼} .

We calculate d̃ν(X), d̃ν(Z) for X ∈ 𝔥, Z ∈ V :

d̃ν(X)φ̃(ḣ, ξ) = ∫_{U} dν(X)φ(ḣ, h · u)e^{−i⟨h·u,ξ⟩}du

= ∫_{U} \frac{d}{dt}φ(exp(−tX)ḣ, exp(−tX)h · u)|_{t=0} · e^{−i⟨h·u,ξ⟩}du

= \frac{d}{dt} ∫_{U} φ(exp(−tX)ḣ, exp(−tX)h · u) · e^{−i⟨exp(−tX)h·u,exp(−tX)^{∗}·ξ⟩}du|_{t=0}

= \frac{d}{dt}φ̃(exp(−tX)ḣ, exp(−tX)^{∗} · ξ)|_{t=0} ;  (6)

d̃ν(Z)φ̃(ḣ, ξ) = ∫_{U} dν(Z)φ(ḣ, h · u)e^{−i⟨h·u,ξ⟩}du

= ∫_{U} \frac{d}{dt}φ(ḣ, −tZ + h · u)|_{t=0} · e^{−i⟨h·u,ξ⟩}du

= ∫_{U} \frac{d}{dt}φ(ḣ, −th · p_{U}(h^{−1} · Z) + h · u)|_{t=0} · e^{−i⟨h·u,ξ⟩}du

(p_{U} denoting projection on U parallel to E)

= \frac{d}{dt} ∫_{U} φ(ḣ, h · u)e^{−i⟨h·u,ξ⟩}e^{−i⟨tZ,ξ⟩}du|_{t=0} (φ ∈ S(E) )

= −i⟨Z, ξ⟩φ̃(ḣ, ξ) .  (7)

Following the notation in lemma 5.2 we define operators Y_{j}, 1 ≤ j ≤ l, on C^{∞}(E^{∗}):

Y_{j}ψ(ḣ, ξ) := ∑_{k=1}^{m} a_{jk}(ξ)d̃ν(X_{k})ψ(ḣ, ξ) .

For f ∈ S(V) we compute:

Y_{j}(˜f̂)(ḣ, ξ) = ∑_{k=1}^{m} a_{jk}(ξ)d̃ν(X_{k})(˜f̂)(ḣ, ξ)

= ∑_{k=1}^{m} a_{jk}(ξ)\frac{d}{dt}(˜f̂)(exp(−tX_{k})ḣ, exp(−tX_{k})^{∗} · ξ)|_{t=0} by (6)

= ∑_{k=1}^{m} a_{jk}(ξ)\frac{d}{dt}f̃(exp(−tX_{k})^{∗} · ξ)|_{t=0} by lemma 4.2

= \frac{d}{dt}f̃(exp(−tY_{j}(ξ))^{∗} · ξ)|_{t=0} = 0 , 1 ≤ j ≤ l .  (8)

We now proceed to a closer study of the operators Y_{j}. A homogeneous polynomial in ξ ∈ V^{∗} is obtained by multiplication and addition of terms of the form ⟨Z, ξ⟩, Z ∈ V . Thus, because of (7) there exist elements P_{jk} ∈ U(V) satisfying d̃ν(P_{jk})ψ(ḣ, ξ) = a_{jk}(ξ)ψ(ḣ, ξ). Using this fact, we obtain:

Y_{j}φ̃(ḣ, ξ) = ∑_{k=1}^{m} a_{jk}(ξ)d̃ν(X_{k})φ̃(ḣ, ξ) = ∑_{k=1}^{m} d̃ν(P_{jk})d̃ν(X_{k})φ̃(ḣ, ξ)

= ∑_{k=1}^{m}[dν(P_{jk})dν(X_{k})φ]^{∼}(ḣ, ξ) ,

so

Y_{j}φ̃ = ∑_{k=1}^{m}[dν(P_{jk})dν(X_{k})φ]^{∼} = [dν(P_{j})φ]^{∼} = d̃ν(P_{j})φ̃ ,

where we have put P_{j} := ∑_{k=1}^{m} P_{jk}X_{k} ∈ U(𝔥 ×_{s} V). Equation (8) now implies:

0 = [dν(P_{j})f̂]^{∼} ⟹ 0 = dν(P_{j})f̂ = [dλ(P_{j})f]^{∧} ⟹ dλ(P_{j})f = 0

by (3) and lemma 5.1.

We have thus proved that there exist elements P_{j} ∈ ker(dλ) such that

Y_{j} = d̃ν(P_{j}) , 1 ≤ j ≤ l .  (9)

Remark . It holds: P_{j} = ∑_{k=1}^{m} X_{k}P_{jk} . In fact, for Q_{j} := ∑_{k=1}^{m} X_{k}P_{jk} we have Q_{j} ∈ K^{1}(𝔥 ×_{s} V) (see section 3), since ad(Z)Q_{j} = ad(Z)P_{j} = 0 for every Z ∈ V (P_{j} ∈ K^{1}(𝔥 ×_{s} V)). It follows: Q_{j} ∈ ker(dλ) ⇒ Q_{j} − P_{j} ∈ ker(dλ) ⇒ Q_{j} − P_{j} = 0, because Q_{j} − P_{j} ∈ U(V) and dλ is injective on U(V).

We come to the main result of this subsection:

Proposition 5.3. For every φ ∈ S_{D}(E) there exists a function F ∈ C^{∞}(V^{∗} \ {0}) such that φ̃(ḣ, ξ) = F (ξ) for ξ ∈ h^{∗} · E^{⊥}, ξ ≠ 0.

Proof. The restriction of π : E^{∗} → V^{∗}, (ḣ, ξ) ↦ ξ, to π^{−1}(V^{∗} \ {0}) is by assumption a submersion. Thus it is enough to show that φ̃ is constant on every subset π^{−1}(ξ), ξ ≠ 0. For this we recall assumption (ii) in the beginning of this section.

Let ξ = h_{1}^{∗} · w_{1} = h_{2}^{∗} · w_{2}, w_{1}, w_{2} ∈ E^{⊥} \ {0}. Then there exists s ∈ H_{E} such that s^{∗} · w_{1} = w_{2}, and we have:

h_{1}^{∗} · w_{1} = (h_{2}s)^{∗} · w_{1} ⇔ h_{1}^{−1}h_{2}s ∈ H_{w_{1}} ⇔ h_{2}s ∈ h_{1}H_{w_{1}} = H_{ξ}h_{1}

and h_{2}sH_{E} = ḣ_{2}, so there is r ∈ H_{ξ} such that r · (ḣ_{1}, ξ) := (rḣ_{1}, r^{∗} · ξ) = (ḣ_{2}, ξ). It therefore remains to show that the function

r ⟼ φ̃(rḣ, r^{∗} · ξ)  (10)

is constant on H_{ξ}.

Relation (9) implies Y_{j}φ̃ = [dν(P_{j})φ]^{∼} = 0, 1 ≤ j ≤ l. On the other hand, with the help of Φ : H × E^{⊥} → ℂ, Φ(h, w) := φ̃(ḣ, h^{∗} · w), we compute:

Y_{j}φ̃(ḣ, ξ) = ∑_{k=1}^{m} a_{jk}(ξ)d̃ν(X_{k})φ̃(ḣ, ξ)

= ∑_{k=1}^{m} a_{jk}(ξ)\frac{d}{dt}φ̃(exp(−tX_{k})ḣ, exp(−tX_{k})^{∗} · ξ)|_{t=0} by (6)

= ∑_{k=1}^{m} a_{jk}(ξ)\frac{d}{dt}Φ(exp(−tX_{k})h, w)|_{t=0} if ξ = h^{∗} · w ,

= \frac{d}{dt}Φ(exp(−tY_{j}(ξ))h, w)|_{t=0} = \frac{d}{dt}φ̃(exp(−tY_{j}(ξ))ḣ, ξ)|_{t=0} .

Since Y_{j}(ξ), 1 ≤ j ≤ l, form a basis of 𝔥_{ξ} for ξ in a dense open set O′ ⊆ V^{∗} \ {0},

(10) is constant on H_{ξ}, ξ ∈ O′. Thus there exists a function F ∈ C^{∞}(O′) satisfying φ̃(ḣ, ξ) = F (ξ), ξ ∈ O′. For 0 ≠ ξ_{0} = h_{0}^{∗} · w_{0} ∉ O′ let W_{(h_{0},w_{0})} be a submanifold of H × E^{⊥} through (h_{0}, w_{0}) which is mapped by (h, w) ↦ h^{∗} · w diffeomorphically onto an open neighborhood W_{ξ_{0}} of ξ_{0} in V^{∗}, and put F (h^{∗} · w) := φ̃(ḣ, h^{∗} · w) for (h, w) ∈ W_{(h_{0},w_{0})}. The argument shows that F can be smoothly extended to an open neighborhood of every ξ ∉ O′, ξ ≠ 0. If we consider a covering of V^{∗} \ {0} by such neighborhoods and by O′, there exists a locally finite subcovering and a partition of unity associated with it, whereby F can be extended to a C^{∞}-function on V^{∗} \ {0}. The equation φ̃ = F ∘ π then holds on the dense subset π^{−1}(O′) and hence everywhere on π^{−1}(V^{∗} \ {0}).

Remark . For the proof we have only used the fact that φ is annihilated by dν(Y), Y ∈ K^{1}(𝔥 ×_{s} V).

Second Step

The proof of F ∈ C^{∞}(V^{∗}) is based on an application of a theorem of Glaeser [5], which we shall now recall.

Definition (Glaeser). Let ω ⊆ ℝ^{n} be an open set, o ⊆ ω, V ⊆ C^{∞}(ω). A function Φ ∈ C^{∞}(ω) is said to be attached to V on o if there is Ψ ∈ V such that Φ − Ψ and all its partial derivatives vanish on o. (In that case Ψ will be called “interpolating function” for Φ). The function Φ will be called bipunctually attached to V if Φ is attached to V on every two-element subset of ω.

Furthermore, we consider an open set Ω ∈ ℝ^{p}, p ≤ n, and a C^{∞}-map θ : ω → Ω. Let A_{θ} denote the set of C^{∞}-functions on ω of the form F ∘ θ, F ∈ C^{∞}(Ω).

Theorem (Glaeser). If θ : ω → Ω satisfies the following 4 conditions and Φ ∈ C^{∞}(ω) is bipunctually attached to A_{θ}, then Φ belongs to A_{θ}.

Θ_{1}: θ : ω → Ω is real analytic.

Θ_{2}: θ has maximal rank (= p) on a dense open subset of ω.

Θ_{3}: θ(ω) is closed in Ω.

Θ_{4}: For every compact set K ⊆ θ(ω) there is a compact set k ⊆ ω such that θ(k) = K.

We prepare the application of this theorem.

Lemma 5.4. There is a compact set K ⊆ H such that the mapping (h, w) ↦ h^{∗} · w from K × E^{⊥} into V^{∗} remains surjective.

Proof. Let S_{V^{∗}} denote the sphere of radius 1 in a norm ∥ · ∥ in V^{∗}. The map σ : H × (E^{⊥} \ \{0\}) → S_{V^{∗}}, (h, w) ↦ \frac{h^{∗}·w}{∥h^{∗}·w∥} is continuous and open, since it is the composition of the submersions (h, w) ↦ h^{∗} · w and v ↦ \frac{v}{∥v∥}. For every ξ = σ(h, w) ∈ S_{V^{∗}} let U_{h} be an open, relatively compact neighborhood of h in H, W_{h} := U_{h} × (E^{⊥} \ {0}). Since S_{V^{∗}} is compact, it is covered by finitely many open sets σ(W_{h_{1}}), …, σ(W_{h_{k}}). The subset K := Ā, A := ∪_{i=1}^{k}U_{h_{i}}, has the required property.

Let K̇ be the projection of K on ^{H}/_{H_{E}}. There is a finite open covering of K̇ by charts (U_{i}, φ_{i}), 1 ≤ i ≤ q, with the property that there exists on every U_{i} a cross section σ_{i} into H. The sets U_{i} can be considered as pairwise disjoint open sets in ℝ^{k} (k = dim ^{H}/_{H_{E}}) (if we identify U_{i} with φ_{i}(U_{i}) and take φ_{i}(U_{i}) bounded, the φ_{i}(U_{i}) can be disjointly distributed in ℝ^{p} by the use of translations).^{3}

Let now ω := (∪_{i=1}^{q}U_{i}) × E^{⊥}, Ω := V^{∗}, θ : ω → V^{∗}, θ(ḣ, w) := σ(ḣ)^{∗} · w, σ|_{U_{i}} := σ_{i}.^{4} The map θ is surjective and satisfies Θ_{2} and Θ_{3} by assumption. Our manifolds has a real analytic atlas, and the cross sections σ_{i} as well as the representation ρ can be taken analytic. This establishes Θ_{1}. The next lemma establishes Θ_{4}.

3This very technical assumption is needed, so as to construct a unique cross section σ from the various σ_{i}. After all, Glaeser’s theorem is formulated for open sets in ℝ^{p} and not in the context of manifolds, as is the case here.

4After choosing bases we can identify E^{⊥} and V^{∗} with ℝ^{n−d} and ℝ^{n} respectively.

Lemma 5.5. For every compact set K′ ⊆ V^{∗} there exists a compact set k ⊆ ω such that θ(k) = K′.

Proof. Similar considerations as those in the proof of lemma 5.4 lead to a compact set K̃ ⊆ ∪_{i=1}^{q}U_{i} with the property that the restriction of θ on K̃ × E^{⊥} is surjective. For ḣ ∈ K̃ we put

Kḣ := {w ∈ E^{⊥}|θ(ḣ, w) ∈ K′} .

Obviously, Kḣ is compact. We finish the proof by verifying the compactness of k := ∪_{ḣ∈K̃}{ḣ} × Kḣ.

Let ( (ḣ_{n}, w_{n}) )_{n∈ℕ} be a sequence in k and suppose that ḣ_{n} converges to ḣ ∈ K̃. We have:

w_{n} ∈ Kḣ_{n} = [(σ(ḣ_{n})^{−1})^{∗} · K′] ∩ E^{⊥} .

Let B be a compact neighborhood of ḣ in ∪_{i=1}^{q}U_{i}. Almost all w_{n} lie in the compact set [(σ(B)^{−1})^{∗} · K′] ∩ E^{⊥}, so there is a subsequence of (w_{n})_{n∈ℕ} which converges to w ∈ [(σ(B)^{−1})^{∗} · K′] ∩ E^{⊥}. Therefore there exists ḣ_{B} ∈ B such that σ(ḣ_{B})^{∗} · w ∈ K′. If we let B shrink to ḣ, we obtain a sequence (ḣ_{B_{n}})_{n∈ℕ} converging to ḣ and having the property that σ(ḣ_{B_{n}})^{∗} · w = θ(ḣ_{B_{n}}, w) ∈ K′. This implies θ(ḣ, w) ∈ K′, so (ḣ, w) ∈ k.

To apply Glaeser’s above theorem to the function Φ on ω, defined by

Φ(ḣ, w) := φ̃(ḣ, θ(ḣ, w)) ,  (11)

we need to show that Φ is bipunctually attached to A_{θ}. Taking into account the goal of the first step, we only have to show this for the subset {(ḣ, 0)|ḣ ∈ ∪_{i=1}^{q}U_{i}} of ω. We first prove an algebraic lemma.

Lemma 5.6. Let P_{k} be a homogeneous polynomial of degree k ≥ 0 on V^{∗}. There is a unique k-linear symmetric form T_{k} on V^{∗} such that T_{k}(x, …, x) = P_{k}(x) for all x ∈ V^{∗}.

Proof. Let (x^{1}, …, x^{n}) denote the vector of coordinates of x ∈ V^{∗} with respect to a fixed basis. The proof follows easily, if we consider the general expression of a symmetric k-linear form:

T_{k}(x_{1}, …, x_{k}) = ∑_{1≤i_{1}≤...≤i_{k}≤n} \frac{1}{k!}c_{i_{1}...i_{k}} ∑_{σ∈S_{k}} x_{σ(1)}^{i_{1}} … x_{σ(k)}^{i_{k}} , c_{i_{1}...i_{k}} ∈ ℂ

(S_{k} the symmetric group of k elements).

Proposition 5.7. The function Φ defined by (11) on ω is attached to A_{θ} on {(ḣ, 0)|ḣ ∈ ∪_{i=1}^{q}U_{i}}.

Proof. Let (u_{1}, …, u_{n−d}) be the vector of coordinates with respect to a basis of U, (w_{1}, …, w_{n−d}) the one with respect to the dual basis of E^{⊥}. We have:

\frac{∂}{∂w′_{j}} Φ(ḣ, w′) = \frac{∂}{∂w′_{j}} ∫_{U} φ(ḣ, σ(ḣ) · u)e^{−i⟨u,w′⟩}du

= −i · ∫_{U} φ(ḣ, σ(ḣ) · u)u_{j}e^{−i⟨u,w′⟩}du .

The moment conditions for φ (proposition 4.4) imply:

(∑_{j=1}^{n−d} w_{j} \frac{∂}{∂w′_{j}})^{m}Φ(ḣ, w′)|_{w′=0} = (−i)^{m}P_{m}(σ(ḣ)^{∗} · w)  (12)

(P_{m} a homogeneous polynomial of degree m on V^{∗}).

For every m ∈ ℤ_{+} let T_{m} be the unique m-linear symmetric form on V^{∗} such that T_{m}(x, …, x) = (−i)^{m}P_{m}(x) (according to the preceding lemma). By the generalized theorem of Emile Borel every (formal) power series is the Taylor series of a C^{∞}-function.^{5} Therefore there exists a function F′ ∈ C^{∞}(V^{∗}) such that D^{m}F′(0) = T_{m}, m ∈ ℤ_{+}. We shall show that F′ ∘ θ is an interpolating function for Φ on {(ḣ, 0)|ḣ ∈ ∪_{i=1}^{q}U_{i}}.

If we set

(ḣ^{∗}T_{m})(w_{1}, …, w_{m}) := T_{m}(σ(ḣ)^{∗} · w_{1}, …, σ(ḣ)^{∗} · w_{m})

and denote by D_{2}^{m}Φ(ḣ, 0) the m-th derivative of w ↦ Φ(ḣ, w) at 0, (12) takes the form

D_{2}^{m}Φ(ḣ, 0)(w, …, w) = (ḣ^{∗}T_{m})(w, …, w) .

Since both sides are symmetric tensors, D_{2}^{m}Φ(ḣ, 0) = ḣ^{∗}T_{m} by the preceding lemma. On the other hand, D_{2}^{m}(F′ ∘ θ)(ḣ, 0) = ḣ^{∗}T_{m}, the left hand side denoting the m-th derivative of w ↦ F′ ∘ θ(ḣ, w) = F′(σ(ḣ)^{∗} · w). Hence,

D_{2}^{m}Φ(ḣ, 0) = D_{2}^{m}(F′ ∘ θ)(ḣ, 0) , m ∈ ℤ_{+} .

Since this last equation holds for all ḣ ∈ ∪_{i=1}^{q}U_{i}, we can apply on both sides a differential operator of constant coefficients in ḣ (∪_{i=1}^{q}U_{i} ⊆ ℝ^{p}) without doing any harm to the equality. This shows that F′ ∘ θ is an interpolating function for Φ.

By Glaeser’s theorem there now exists F′ ∈ C^{∞}(V^{∗}) such that Φ and F′ ∘θ coincide on ω. Therefore, the function F of the first step can be extended to the C^{∞}-function F′ on V^{∗}. We shall continue to write F instead of F′. By continuity we have φ̃(ḣ, ξ) = F (ξ) for all ξ ∈ h^{∗} · E^{⊥}, ḣ ∈ ^{H}/_{H_{E}}.

Third step

Proposition 5.8. The function F defined above belongs to the Schwartz class S(V^{∗}).

5This theorem is a special case of theorem I in [16, page 65].

Proof. It remains to show that for every p ∈ ℤ_{+} and every differential operator D with constant coefficients on V^{∗}

sup_{ξ∈V^{∗}} ∥ξ∥^{p}|DF (ξ)| < ∞ .

Let K ⊆ H be as in lemma 5.4, K̇ the projection of K on ^{H}/_{H_{E}} and (U_{i})_{1≤i≤q} a covering of K̇ by open, relatively compact sets, which admit local cross sections σ_{i} on their topological closures. Thus we have to show that

sup_{ḣ∈K̇,w∈E^{⊥}} ∥σ_{i}(ḣ)^{∗} · w∥^{p} · |DF (σ_{i}(ḣ)^{∗} · w)| < ∞  (13)

(if ḣ is contained in U_{i}). For this we shall express D in coordinates (ḣ, w).

Let us take a basis of E^{⊥}, extend it to one of V^{∗} and denote by (w_{1}, …, w_{n−d}) and (ξ_{1}, …, ξ_{n}) the vectors of coordinates on E^{⊥} and V^{∗} respectively. The automorphisms ρ^{∗}(σ_{i}(ḣ)) can then be regarded as n × n-matrices. Putting θ_{i} : U_{i} ×E^{⊥} → V^{∗}, θ_{i}(ḣ, w) := σ_{i}(ḣ)^{∗} ·w, 1 ≤ i ≤ q, we have (X_{1}, …, X_{m} a basis of the Lie algebra 𝔥 of H):

\frac{∂}{∂w_{j}} F ∘ θ_{i}(ḣ, w) = ∑_{l=1}^{n} \frac{∂F}{∂ξ_{l}} (σ_{i}(ḣ)^{∗} · w) · ρ^{∗}(σ_{i}(ḣ))_{lj} , 1 ≤ j ≤ n − d ,

X_{k}(F ∘ θ_{i})(ḣ, w) := \frac{d}{dt}F ∘ θ_{i}(exp(tX_{k})ḣ, w)|_{t=0}

= ∑_{l=1}^{n} \frac{∂F}{∂ξ_{l}} (σ_{i}(ḣ)^{∗} · w) · \frac{d}{dt}[ρ^{∗}(σ_{i}(exp(tX_{k})ḣ))w]_{l}|_{t=0} , 1 ≤ k ≤ m .

We consider these equations as a linear system in \frac{∂F}{∂ξ_{1}} , …, \frac{∂F}{∂ξ_{n}}. For every (ḣ_{0}, w_{0}) ∈ K̇ × (E^{⊥} \ {0}) there is an index i such that ḣ_{0} ∈ U_{i}, and we can choose n of the above equations such that the determinant of the coefficients of this part of the system does not vanish (θ_{i} is on U_{i} × (E^{⊥} \ {0}) a submersion). Since this determinant is a homogeneous polynomial in w, it remains non-zero on a neighborhood of (ḣ_{0}, w_{0}) of the form O × ℝ^{∗} · W, where O is a neighborhood of ḣ_{0} in U_{i} and W one of \frac{w_{0}}{∥w_{0}∥} in the sphere S_{E^{⊥}} of radius 1 in E^{⊥}. Using Cramer’s rule, we obtain:

\frac{∂F}{∂ξ_{l}} (σ_{i}(ḣ)^{∗} · w) = ∑_{j=1}^{n−d} f_{j}(ḣ, w)\frac{∂}{∂w_{j}} F ∘ θ_{i}(ḣ, w) + ∑_{k=1}^{m} g_{k}(ḣ, w)X_{k}(F ∘ θ_{i})(ḣ, w) ,

where f_{j}, g_{k} are smooth homogeneous rational functions in w (f_{j} of degree 0, g_{k} of degree −1) (take f_{j}, g_{k} to be zero, if a term does not appear). Repeating the procedure with \frac{∂F}{∂ξ_{l}} instead of F, we obtain an expression for the second derivatives of F, etc. Eventually this leads to an expression of the operator D in coordinates (ḣ, w):

DF (σ_{i}(ḣ)^{∗} · w) = ∑_{j} e_{j}(ḣ, w)T_{j}P_{j}(\frac{∂}{∂w_{1}} , …, \frac{∂}{∂w_{n−d}})F ∘ θ_{i}(ḣ, w)

= ∑_{j} e_{j}(ḣ, w)T_{j}P_{j}(\frac{∂}{∂w_{1}} , …, \frac{∂}{∂w_{n−d}})φ̃(ḣ, σ_{i}(ḣ)^{∗} · w) .  (14)

Therein T_{j} is a differential operator on ^{H}/_{H_{E}}, P_{j}(\frac{∂}{∂w_{1}} , …, \frac{∂}{∂w_{n−d}}) a polynomial in \frac{∂}{∂w_{1}} , …, \frac{∂}{∂w_{n−d}}, and e_{j} smooth rational functions, homogeneous in w of degree ≤ 0.

Relation (13) is a property for w in a neighborhood of infinity. For ∥w∥ ≤ 1 we actually have ∥σ_{i}(ḣ)^{∗} · w∥ ≤ ∥ρ^{∗}(σ_{i}(ḣ))∥, and

A_{i} := sup_{ḣ∈U_{i}} ∥ρ^{∗}(σ_{i}(ḣ))∥ < ∞ ,

since U_{i} is relatively compact and σ_{i} a local cross section on Ū_{i}. The region of V^{∗} we obtain by σ_{i}(ḣ)^{∗} · w (1 ≤ i ≤ q) from ḣ ∈ K̇, ∥w∥ ≥ 1, contains {ξ ∈ V^{∗}| ∥ξ∥ > max_{1≤i≤q} A_{i}}. Therefore we can restrict ourselves to ∥w∥ ≥ 1 in (13).

Equation (14) holds for (ḣ, w) ∈ O × ℝ^{∗} · W. If we take O and W to be compact, all functions e_{j} are bounded on O × (ℝ\] − 1, 1[) · W, and (13) holds for (ḣ, w) ∈ O × (ℝ\] − 1, 1[) · W, because φ̃ is a rapidly decreasing function (cf. section 2). Due to the compactness of K̇ and S_{E^{⊥}}, we now only need finitely many sets of the form O × (ℝ\] − 1, 1[) · W to obtain a full neighborhood of the infinity in V^{∗}. The proof is thereby completed.

6. The image in the case d < n − 1

Richter and Kurusa have shown that in the classical case (H = SO(n), V = ℝ^{n}) and for d < n − 1 the range-characterizing partial differential equations (which are equivalent to dν(T)φ = 0 for T ∈ K^{1}(so(n) ×_{s} ℝ^{n}), see [14, page 72ff]) suffice to characterize the range S(ℝ^{n})^{∧} (the latter author derives directly the moment conditions of Helgason) [11], [13]. Gonzalez has given a group theoretical proof [6]. The fact is also true in the general situation, provided that the set of planes in E (actually in E^{∗}) is sufficiently rich.

Definition 6.1. A collection U of k-dimensional linear subspaces of V^{∗} will be called fundamental system if every homogeneous polynomial on V^{∗} is completely determined by its restriction on the union of all elements of U.

For instance, a set of hyperplanes (k = n − 1) is a fundamental system iff it contains infinitely many elements.

We proceed to the following assumption (denoted in the sequel by (G)) on the action of the group H:

There is a dense set A ⊆ V^{∗}\{0} with the following property: For every ξ ∈ A the planes of the form h^{∗} · E^{⊥} through ξ form a fundamental system.

The set of ξ ∈ V^{∗} \ {0} for which the statement in (G) holds is ρ^{∗}(H)- invariant. Therefore we only have to establish (G) for a dense set in E^{⊥} \ {0}. This makes things much easier in concrete situations.

Proposition 6.2. Assumption (G) implies: S(V)^{∧} = S_{D}(E).

Proof. Let φ ∈ S_{D}(E), φ̃ the Fourier transform. We denote by D_{2}^{m}φ̃(ḣ, ξ_{0}) the m-th derivative of ξ ↦ φ̃(ḣ, ξ) (as a function on h^{∗} ·E^{⊥}) at the point ξ_{0} ∈ h^{∗} ·E^{⊥}. The proof will be given by establishing the moment conditions for φ. These are equivalent to

D_{2}^{m}φ̃(ḣ, 0) · ξ^{⊗^{m}} = (−i)^{m}P_{m}(ξ) , ξ ∈ h^{∗} · E^{⊥} ,

P_{m} homogeneous polynomials of degree m ≥ 0 on V^{∗} (cf. proposition 4.4).

Let Y_{j}(ξ), 1 ≤ j ≤ l, be a basis of 𝔥_{ξ} for ξ in a dense open subset of V^{∗} \ {0} (notation as in section 5). It follows from the considerations in section 5 that for 1 ≤ j ≤ l the function

(t, s) ⟼ φ̃(exp(−tY_{j}(ξ))ḣ, sξ)

does not depend on t, nor does the m-th partial derivative with respect to s at s = 0:

D_{2}^{m}φ̃(exp(−tY_{j}(ξ))ḣ, 0) · ξ^{⊗^{m}} .

Following the reasoning of section 5, there exists a function F_{m} ∈ C^{∞}(V^{∗} \ {0}) such that

D_{2}^{m}φ̃(ḣ, 0) · ξ^{⊗^{m}} = F_{m}(ξ)  (15)

for ξ ∈ h^{∗} · E^{⊥}, ξ ≠ 0. Let ξ_{0} ∈ A, A as in (G). The map

ξ ⟼ D^{k}F_{m}(ξ_{0}) · ξ^{⊗^{k}} , k > m ,

(D^{k}: the k-th derivative) is a homogeneous polynomial, whose restriction on every plane h^{∗} · E^{⊥} containing ξ_{0} is equal to zero by (15). The fact ξ_{0} ∈ A implies D^{k}F_{m}(ξ_{0})·ξ^{⊗^{k}} ≡ 0, and by lemma 5.6 D^{k}F_{m}(ξ_{0}) = 0. Since A is dense, we obtain D^{k}F_{m} = 0 for k > m. Therefore F_{m} equals a polynomial (−i)^{m}P_{m} of degree ≤ m on V^{∗} \ {0} and can be extended to it on V^{∗}. By continuity,

D_{2}^{m}φ̃(ḣ, 0) · ξ^{⊗^{m}} = (−i)^{m}P_{m}(ξ) , ξ ∈ h^{∗} · E^{⊥} .

Hereby we conclude that P_{m} is homogeneous of degree m, and the proof is completed.

However, assumption (G) is not necessary for a relation of the form S(V)^{∧} = S_{D}(E), as the last example in section 9 shows.

7. The Radon transform (d = n − 1)

In this section we focus our attention on the special case d = n − 1. As it will come out, this leads to the classical Radon transform. Different situations can only appear, if we remove some of our conditions on the group action, or if we completely change the setting and allow for E sets of planes which do not admit a transitive group action. For the latter possibility the reader is referred to [2].

Let H be a Lie group, ρ a representation of H on a (finite-dimensional) vector space V . The contragredient representation will be denoted by ρ^{∗}. We fix a linear subspace E ⊆ V of codimension 1 and denote by E^{⊥} ⊆ V^{∗} its (onedimensional) orthogonal complement. We make the following assumptions:

1. ρ(H) consists of unimodular automorphisms of V , and the same holds for ρ(H_{E}) (the isotropy subgroup of E) in the case of E.

2. The map from H×E^{⊥} into V^{∗}, (h, w) ↦ ρ^{∗}(h)w, is a surjective submersion.^{6}

We put h · u := ρ(h)u, h^{∗} · w := ρ^{∗}(h)w. Let ∥ · ∥ be a norm on V^{∗}, S_{V^{∗}} the sphere of radius 1. The group H acts transitively on S_{V^{∗}}:

h · w := \frac{h^{∗} · w}{∥h^{∗} · w∥} .

Let w_{0} ∈ S_{V^{∗}} ∩ E^{⊥}, H_{w_{0}}^{S} the isotropy subgroup of w_{0}. Since H_{E} operates by

unimodular automorphisms on E^{⊥}, we obtain H_{E}^{o}⊆ H_{w_{0}}^{S} ⊆ H_{E} (H_{E}^{o} denoting the connected component of H_{E}) and H_{w_{0}}^{S} = H_{w_{0}} (the isotropy subgroup in the contragredient action). The transitivity of the action on S_{V^{∗}} implies H_{ξ}^{S} = H_{ξ} for every ξ ∈ S_{V^{∗}}. Thus, ^{H}/_{H_{ξ}} ≅ S_{V^{∗}} for ξ ∈ V^{∗} \ {0}. The orbit ρ^{∗}(H)ξ is bounded because ^{H}/_{H_{ξ}} is compact. Since this holds for every ξ, ρ^{∗}(H) itself is bounded (in the vector space L(V^{∗}) of all endomorphisms of V^{∗}). Therefore, det[ρ^{∗}(H)] is a bounded subgroup of ℝ^{∗}, so det[ρ^{∗}(H)] ⊆ {−1, 1}. This relation also holds for the closure \overline{ρ^{∗}(H)} in L(V^{∗}). Thus, \overline{ρ^{∗}(H)} is a compact subgroup of GL(V^{∗}) and hence leaves a positive definite inner product invariant. With the help of it, V^{∗} can be identified with V^{∗∗} ≅ V , and we have ρ^{∗} = ρ. Eventually, V ≅ V^{∗} is identified with ℝ^{n} with the help of an orthonormal basis. We obtain:

ρ(H) is a group of orthogonal matrices and operates transitively on the sphere S^{n−1}. It holds ρ = ρ^{∗}, and E ⊆ ℝ^{n} is a subspace of codimension 1.

This is the situation in the classical case.

As for the range of S(V) under f ↦ f̂, it is completely characterized by the moment conditions (see [9, page 100]):

Proposition 7.1. In the case d = n − 1 we have: S(V)^{∧} = S_{M}(E).

Proof. Since ξ ∈ h · E^{⊥}, ξ ≠ 0, completely determines ḣ = hH_{E} (hH_{E} ⇔ h · E = ξ^{⊥}), we deduce at once the result of the first step in section 5. The differential operators are not needed for the second and third step.^{7}

8. The range of C_{c}^{∞}(V)

The characterization of the range of C_{c}^{∞}(V), the set of C^{∞}-functions with compact support, under the d-plane transform will be done with the help of a modified version of the Paley-Wiener theorem, due to Helgason [8], which is recalled first. Condition (ii) in the beginning of section 5 can be dropped throughout this section.

Let D_{A}(ℝ^{n}) denote the class of C^{∞}-functions on ℝ^{n} with support in {x| |x| ≤ A}.

6Notice that condition (ii) in the beginning of section 5 here degenerates.

7Obviously, we have here dν(Y) = 0 for Y ∈ K^{1}(𝔥 ×_{s} V).

Theorem . The Fourier transform f ↦ f̃ maps D_{A}(ℝ^{n}) onto the set of functions f̃(λω) = φ(λ, ω) ∈ C^{∞}(ℝ × S^{n−1}) satisfying:

(i) For each ω, the function λ ↦ φ(λ, ω) extends to a holomorphic function on

ℂ such that

sup_{λ,ω} |φ(λ, ω)λ^{N}e^{−A|Imλ|}| < ∞

for each integer N ∈ ℤ_{+}.

(ii) For each k ∈ ℤ_{+} and every vector a = (a_{1}, …, a_{n}) ∈ ℂ^{n} satisfying ∑_{i=1}^{n} a_{i}^{2} =

0 the function

λ ↦ λ^{−k} ∫_{S^{n−1}} φ(λ, ω)⟨a, ω⟩^{k}dω

is even and holomorphic on ℂ.

For a proof we refer to [9, page 23ff].

Before we state the main theorem, we need some facts about spherical harmonics.

Proposition . The eigenspaces of the Laplacian on S^{n−1} are of the form

E_{k} = span{f_{a,k}|f_{a,k}(ω) = ⟨a, ω⟩^{k}, ω ∈ S^{n−1}} ,

where a = (a_{1}, …, a_{n}) ∈ ℂ^{n} satisfies ∑_{i=1}^{n} a_{i}^{2} = 0, and k ∈ ℤ_{+}. The subspaces E_{k} are pairwise orthogonal in L^{2}(S^{n−1}). If P_{k} denotes the set of restrictions on S^{n−1} of the homogeneous polynomials of degree k on ℝ^{n}, it holds

P_{k} = E_{k} + E_{k−2} + … + E_{k−2m} , m = [\frac{k}{2}] .

For a proof see [9, page 17ff].

Let ⟨·, ·⟩ be a (positive definite) inner product on V such that E and U are orthogonal. S_{V} stands for the sphere of radius 1 in the induced norm. For h ∈ H we put

τ(h) := ∥ρ(h^{−1})∥ ,

ρ being the representation of H on V and ∥ · ∥ the operator norm. The map τ is continuous. For u ∈ U, ∥u∥ ≥ τ(h), h · (u + E) has at most one point with S_{V} in common. If suppf ⊆ {x ∈ V | ∥x∥ ≤ A}, we therefore have f̂(ḣ, h · u) = 0 for ∥u∥ ≥ Aτ(h).

Theorem 8.1. φ ∈ S_{M}(E) is the d-plane transform of a C^{∞}-function f of compact support on V iff there exists a continuous function c on H such that for every h ∈ H φ(ḣ, h · u) = 0 for ∥u∥ ≥ c(h).

Proof. We only have to show that the condition is sufficient. We calculate:

φ̃(ḣ, ξ) = ∫_{U} φ(ḣ, h · u)e^{−i⟨h·u,ξ⟩}du = ∫_{U} φ(ḣ, h · u) ∑_{m=0}^{∞} \frac{(−i)^{m}}{m!} ⟨h · u, ξ⟩^{m}du

= ∑_{m=0}^{∞} \frac{(−i)^{m}}{m!} ∫_{U} φ(ḣ, h · u)⟨h · u, ξ⟩^{m}du

(the region of integration is bounded)

= ∑_{m=0}^{∞} \frac{(−i)^{m}}{m!} P_{m}(ξ)

by the moment conditions. The last series converges for all ξ ∈ h^{∗} · E^{⊥}, and, since it is a power series, for all ξ ∈ V^{∗}. It defines an analytic function F.

The spaces V and V^{∗} can be identified with the help of the inner product. We choose an orthonormal basis and identify both spaces with ℝ^{n}.

F (ξ) = ∑_{m=0}^{∞} \frac{(−i)^{m}}{m!} P_{m}(ξ)

is then an analytic function on ℝ^{n} and can be extended to a holomorphic function on ℂ^{n} by the above power series.

Let F (λω) =: ψ(λ, ω) ∈ C^{∞}(ℝ × S^{n−1}). We show that ψ satisfies the two conditions of the theorem in the beginning.

Ad (i): It is enough to verify the relation for even exponents N. We calculate (ω = h^{∗} · w ∈ S^{n−1}, w ∈ U):

|ψ(λ, ω)λ^{2N}| = |φ̃(ḣ, λω)λ^{2N}| = | ∫_{U} φ(ḣ, h · u)λ^{2N}e^{−i⟨h·u,λh^{∗}·w⟩}du|
= | ∫_{U} φ(ḣ, h · u)λ^{2N}e^{−iλ⟨u,w⟩}du|

= | ∫_{U} φ(ḣ, h · u)(− ∑_{k=1}^{n−d} \frac{∂^{2}}{∂u_{k}^{2}} )^{N}e^{−iλ⟨u,w⟩} · ∥w∥^{−2N}du|

(taking such an orthonormal basis that the first n − d vectors belong to U)

= ∥w∥^{−2N}| ∫_{U}(− ∑_{k=1}^{n−d} \frac{∂^{2}}{∂u_{k}^{2}} )^{N}φ(ḣ, h · u)e^{−iλ⟨u,w⟩}du|

≤ ∥w∥^{−2N} ∫_{U} |(∑_{k=1}^{n−d} \frac{∂^{2}}{∂u_{k}^{2}} )^{N}φ(ḣ, h · u)|du · e^{c(h)∥w∥|Imλ|} .

For τ^{∗}(h) := ∥ρ^{∗}(h^{−1})∥ we have: \frac{1}{τ^{∗}(h^{−1})} ≤ ∥w∥ ≤ τ^{∗}(h), so

|ψ(λ, ω)λ^{2N}| ≤ τ^{∗}(h^{−1})^{2N} ∫_{U} |(∑_{k=1}^{n−d} \frac{∂^{2}}{∂u_{k}^{2}} )^{N}φ(ḣ, h · u)|du · e^{c(h)τ^{∗}(h)|Imλ|} .

Let K ⊆ H be a compact subset with the property of lemma 5.4. Putting

C_{N} := sup_{h∈K} τ^{∗}(h^{−1})^{2N} ∫_{U} |(∑_{k=1}^{n−d} \frac{∂^{2}}{∂u_{k}^{2}} )^{N}φ(ḣ, h · u)|du , A := sup_{h∈K} c(h)τ^{∗}(h)

we obtain the desired relation

|ψ(λ, ω)λ^{2N}| ≤ C_{N}e^{A|Imλ|} .

Ad (ii): Since ∑_{m=0}^{∞} \frac{(−i)^{m}}{m!} P_{m}(ξ) converges uniformly on compact sets, we have:

∫_{S^{n−1}} ψ(λ, ω)⟨a, ω⟩^{k}dω = ∫_{S^{n−1}} F (λω)⟨a, ω⟩^{k}dω

= ∫_{S^{n−1}} ∑_{m=0}^{∞} \frac{(−i)^{m}}{m!} P_{m}(λω)⟨a, ω⟩^{k}dω

= ∑_{m=0}^{∞} ∫_{S^{n−1}} \frac{(−i)^{m}}{m!} P_{m}(λω)⟨a, ω⟩^{k}dω = ∑_{m=0}^{∞} \frac{(−iλ)^{m}}{m!} ∫_{S^{n−1}} P_{m}(ω)⟨a, ω⟩^{k}dω .

The proposition above implies that ∫_{S^{n−1}} P_{m}(ω)⟨a, ω⟩^{k}dω vanishes, unless m−k ∈ ℤ_{+}. This verifies (ii) .

Concluding, there exists a function f ∈ C_{c}^{∞}(V) such that

f̃(λω) = ψ(λ, ω) = F (λω) = φ̃(ḣ, λω) , ω ∈ h^{∗} · U .

Since on the other hand f̃(λω) = ˜ f̂(ḣ, λω), we obtain φ = f̂ by the inverse Fourier transform.

Remark . The application of the modified version of the Paley-Wiener theorem in the above proof shows that the support of f is contained in the ball of radius A, which depends on the function c(h) and the compact set K. In the classical case the theorem reduces to theorem 2.10 in [9, page 109], since H is compact. Furthermore, for functions f of the Schwartz class, we obtain Helgason’s support theorem [9, page 105] by taking c(h) ≡ A.

9. Examples

1. We begin by a counterexample, which shows that condition (i) in the beginning of section 5 cannot be weakened substantially.

Let

H = \{( \begin{matrix}1 & a & c \\ 0 & 1 & b \\ 0 & 0 & 1 \end{matrix} )| a, b, c ∈ ℝ\} ,

V = ℝ^{3}, E = ⟨(0, 0, 1)⟩, U = ⟨(1, 0, 0), (0, 1, 0)⟩ ≅ E^{⊥} via the standard scalar product. The assumptions we have made in the general case are all satisfied except of (i), section 5. The vector (0, 0, 1) is not contained in the image (which is dense in ℝ^{3}) of the mapping

(a, b, c, w_{1}, w_{2}) ⟼ ( \begin{matrix}1 & a & c \\ 0 & 1 & b \\ 0 & 0 & 1 \end{matrix} )^{∗} · ( \begin{matrix}w_{1} \\ w_{2} \\ 0 \end{matrix} ) = ( \begin{matrix}w_{1} \\ −aw_{1} + w_{2} \\ (ab − c)w_{1} − bw_{2} \end{matrix} ) .

Let g ∈ C_{c}^{∞}(ℝ), g nowhere zero on [−1, 1], and define F : ℝ^{3} → ℝ by

F (ξ_{1}, ξ_{2}, ξ_{3}) := g(ξ_{1}^{2} + ξ_{2}^{2} + ξ_{3}^{2}) · g(\frac{1}{ξ_{1}^{2} + ξ_{2}^{2} + ξ_{3}^{2}} ) · \frac{1}{ξ_{1}^{2} + ξ_{2}^{2}} .

Then ψ(ḣ, ξ) := F (ξ), ξ ∈ h^{∗} · E^{⊥}, belongs to S(E^{∗}) and hence there exists φ ∈ S(E) such that φ̃ = ψ. Due to the construction of ψ it holds φ ∈ S_{D}(E) ∩ S_{M}(E). However, there is no f ∈ S(ℝ^{3}) such that f̂ = φ, because otherwise we would have f̃ = F, which is a contradiction, since F does not extend to a C^{∞}-function on ℝ^{3}.

2. The action of H = SO(n) on V = ℝ^{n} (by matrix multiplication) leads to the classical d-plane transform. For the range of S(ℝ^{n}) the reader is referred to [13] and [6].

3. H = SO(p, n − p) operates on V = ℝ^{n} by matrix multiplication. We introduce the following basis on the Lie algebra so(p, n − p) (E_{ij} denotes the matrix (δ_{ik}δ_{jl})_{k,l}):

\begin{matrix}X_{ij} := E_{ij} − E_{ji} & for & 1 ≤ i < j ≤ p , p + 1 ≤ i < j ≤ n , \\ Y_{ij} := E_{ij} + E_{ji} & for & 1 ≤ i ≤ p , p + 1 ≤ j ≤ n . \end{matrix}

We agree to write X_{ji} for −X_{ij} and Y_{ji} for Y_{ij} (in the above range of the indices). Z_{1}, …, Z_{n} will denote the standard basis of ℝ^{n}.

The Lie algebra so(n) ×_{s} ℝ^{n} is isomorphic to the Lie algebra of all (n + 1) × (n + 1)-matrices of the form

( \begin{matrix}T & Z \\ 0 & 0 \end{matrix} ) , T ∈ so(n) , Z ∈ ℝ^{n} .

Similarly, so(p, n−p)×_{s}ℝ^{n} is isomorphic to the Lie algebra of the (n+1)×(n+1)- matrices of the form

( \begin{matrix}T_{1} & X & U_{1} \\ X^{T} & T_{2} & U_{2} \\ 0 & 0 & 0 \end{matrix} ) ,

T_{1} ∈ so(p), T_{2} ∈ so(n − p), U_{1} ∈ ℝ^{p}, U_{2} ∈ ℝ^{n−p} and X an arbitrary p × (n − p)- matrix. The complexification of so(n) ×_{s} ℝ^{n} is isomorphic to the complexification of so(p, n − p) ×_{s} ℝ^{n} by the mapping

( \begin{matrix}X_{1} & X_{2} & U_{1} \\ −X_{2}^{T} & X_{3} & U_{2} \\ 0 & 0 & 0 \end{matrix} ) ⟼ ( \begin{matrix}X_{1} & iX_{2} & U_{1} \\ iX_{2}^{T} & X_{3} & −iU_{2} \\ 0 & 0 & 0 \end{matrix} ) ,

where X_{1} ∈ so(p, ℂ), X_{3} ∈ so(n − p, ℂ), U_{1} ∈ ℂ^{p}, U_{2} ∈ ℂ^{n−p} and X_{2} is an arbitrary p × (n − p)-matrix. With the help of this isomorphism we obtain from [14, page 62ff] a basis V_{ijl}, 1 ≤ i < j < l ≤ n, of K^{1}(so(p, n − p) ×_{s} ℝ^{n}):

V_{ijl} = \{ \begin{matrix}X_{ij}Z_{l} + X_{jl}Z_{i} + X_{li}Z_{j} & for & l ≤ p or i ≥ p + 1 \\ X_{ij}Z_{l} − Y_{jl}Z_{i} + Y_{li}Z_{j} & for & j ≤ p < l \\ Y_{ij}Z_{l} + X_{jl}Z_{i} − Y_{li}Z_{j} & for & i ≤ p < j \end{matrix} .

The dual space of V = ℝ^{n} will be identified with ℝ^{n} via

B(x, y) := ∑_{i=1}^{p} x_{i}y_{i} − ∑_{i=p+1}^{n} x_{i}y_{i} .

Since H leaves B invariant, ρ = ρ^{∗}.

Let E := ⟨Z_{k}, Z_{k+1}, …, Z_{k+d−1}⟩, 1 < k ≤ k + d − 1 < n, that is, 1 < k < n + 1 − d (in particular, d < n − 1), U := E^{⊥}. Conditions (i) and (ii) in the beginning of section 5 are satisfied.

We follow [13] and introduce a system of coordinates on a neighborhood of E in E. Let G(d, n) denote the set of d-planes in ℝ^{n}, G_{d,n} ⊆ G(d, n) the subset of d-dimensional subspaces. For every multiindex j = (j_{1}, …, j_{d}), 1 ≤ j_{1} < … < j_{d} ≤ n, let

U_{j} := \{σ ∈ G_{d,n} | \begin{matrix}the orthogonal projection of σ on the subspace \\ spanned by Z_{j_{1}}, …, Z_{j_{d}} is bijective \end{matrix} \} .

Let 1 ≤ j_{d+1} < … < j_{n} ≤ n be such that {j_{d+1}, …, j_{n}} ∪ {j_{1}, …, j_{d}} = {1, …, n}. Let σ ∈ U_{j}, v_{j_{α}} ∈ σ be the inverse image of Z_{j_{α}} under the orthogonal projection onto span{Z_{j_{1}}, …, Z_{j_{d}}} (1 ≤ α ≤ d). It holds:

V_{j_{α}} = Z_{j_{α}} + ∑_{κ=d+1}^{n} σ_{j_{κ}j_{α}}Z_{j_{κ}}

with real coefficients σ_{j_{κ}j_{α}}. In what follows, the range of α, κ will always be [1, d], [d + 1, n] respectively. We define the n × d-matrix Σ by taking v_{j_{1}}, …, v_{j_{d}} for its columns (expressed in the basis Z_{1}, …, Z_{n}). The open set U_{j} can now be parametrized by Σ, that is, by (σ_{j_{κ}j_{α}})_{κ,α}. Since ∪_{j}U_{j} = G_{d,n}, we obtain in this way an atlas for G_{d,n}.

The space G(d, n) is a vector bundle over G_{d,n}. Let π : G(d, n) → G_{d,n} denote the projection. We take ξ ∈ G(d, n) and put σ := π(ξ) ∈ G_{d,n}. If σ ∈ U_{j}, {y} := ξ ∩ ⟨Z_{j_{d+1}}, …, Z_{j_{n}}⟩, it holds: ξ = y + σ. We agree to write y = ∑_{κ=d+1}^{n} y_{j_{κ}}Z_{j_{κ}} and parametrize π^{−1}(U_{j}) by the vector

( (σ_{j_{κ}j_{α}})_{κ,α}, y_{j_{d+1}}, …, y_{j_{n}}) ∈ ℝ^{d(n−d)} × ℝ^{n−d} .  (16)

However, not every d-plane in π^{−1}(U_{j}) can be found in E. In fact, π(E) consists precisely of the subspaces σ, where the signature of B equals its signature on E. We conclude that E ⊆ G(d, n) is open, so we can restrict the above parametrization to π^{−1}(U_{j}) ∩ E (and obtain in this way an atlas of E).

The action of SO(p, n − p) on ℝ^{n} possesses property (G) of section 6, and consequently the differential operators dν(V_{ijl}) characterize the range of S(ℝ^{n}). However, the computation of these operators in the coordinates (16) is very lengthy and tedious (see [15, page 36-38]). Their second order terms are linear combinations of

T_{j_{κ}j_{α}j_{λ}} := \frac{∂^{2}}{∂σ_{j_{κ}j_{α}}∂y_{j_{λ}}} − \frac{∂^{2}}{∂σ_{j_{λ}j_{α}}∂y_{j_{κ}}} ,

their first order ones consist of derivatives in the y-coordinates, and the coefficients of all terms only depend on σ_{j_{κ}j_{α}}. Nevertheless, we can prove the following result, which is analogous to that of Richter for the classical case [14, page 73ff]:

Proposition 9.1. A function φ ∈ S(E) satisfies dν(V_{ijl})φ = 0 for 1 ≤ i < j < l ≤ n iff for every multiindex j = (j_{1}, …, j_{d}) as above, φ_{0} satisfies T_{j_{κ}j_{α}j_{λ}}φ_{0} = 0 for 1 ≤ α ≤ d, d + 1 ≤ κ, λ ≤ n on π^{−1}(U_{j}) ∩ E, where φ_{0} is a modified function, defined with the help of the n × d-matrix Σ:

φ_{0}(Σ, y) := | det Σ^{T} I_{p,n−p}Σ|^{− \frac{1}{2}} φ(Σ, y)

(I_{p,n−p} denoting the matrix of B in the basis Z_{1}, …, Z_{n}).

Proof. — The condition is necessary: If φ ∈ S_{D}(E), there exists f ∈ S(ℝ^{n}) such that f̂ = φ. By looking at the integration measure on the various d-planes f is integrated on, we conclude that

φ_{0}(Σ, y) = ∫_{ℝ^{d}} f(Σx + y)dx_{1} … dx_{d} , x = (x_{1}, …, x_{d})^{T} .

The relations T_{j_{κ}j_{α}j_{λ}}φ_{0} = 0 now follow immediately.

— The condition is sufficient: We compute the Fourier transform of φ (A ∈ SO(p, n − p), u ∈ U):

φ̃(A · E, Au) = ∫_{U} φ(A · E, Au′)e^{−i⟨Au,Au′⟩}du′ = ∫_{U} φ(Σ, y)e^{−i⟨Au,y⟩}du′

in the notation of (16),

= | det Σ^{T} I_{p,n−p}Σ|^{− \frac{1}{2}} ∫_{ℝ^{n−d}} φ(Σ, y)e^{−i⟨Au,y⟩}dy_{j_{d+1}} … dy_{j_{n}}
= ∫_{ℝ^{n−d}} φ_{0}(Σ, y)e^{−i⟨Au,y⟩}dy_{j_{d+1}} … dy_{j_{n}} .

In a completely analogous way to that of Richter [13, page 252-254] it can now be shown that φ̃(A · E, Au) only depends on Au, so there exists a function F ∈ C^{∞}(ℝ^{n} \ {0}) such that φ̃(A · E, Au) = F (Au). The proof now follows by remark 2 to lemma 5.2 and the subsequent computation of the Fourier transforms.

4. The group of orthogonal matrices

H = \{( \begin{matrix}x_{1} & −x_{2} & −x_{3} & −x_{4} \\ x_{2} & x_{1} & x_{4} & −x_{3} \\ x_{3} & −x_{4} & x_{1} & x_{2} \\ x_{4} & x_{3} & −x_{2} & x_{1} \end{matrix} )| x_{1}^{2} + x_{2}^{2} + x_{3}^{2} + x_{4}^{2} = 1\} < SO(4)

operates on V = ℝ^{4}, which is identified with (ℝ^{4})^{∗} via the canonical scalar product. We have ρ = ρ^{∗}.

The choice E = ⟨Z_{1}⟩ (notation as above) shows how indispensable assumption (ii) , section 5, is for our considerations. The other assumptions are all satisfied. For ξ ≠ 0 we have H_{ξ} = {e}. Thus, there should be no need of differential operators. But since every point ξ lies on more than one hyperplanes of the form h · E^{⊥}, the moment conditions fail to characterize the range of S(ℝ^{4}). A counterexample can be constructed as in [6, page 605].

Now let E = ⟨Z_{1}, Z_{2}⟩. All assumptions are satisfied. The action does not have property (G) of section 6. We have instead the extreme situation where two planes of the form h · E^{⊥} either coincide or are complementary. Here we cannot dispense with the moment conditions. Otherwise, S(ℝ^{4})^{∧} = S(E) would hold, but this is contradicted by the following counterexample:

Let g ∈ C_{c}^{∞}(ℝ^{4}), g(0) ≠ 0. Let (x_{1}, x_{2}, x_{3}, x_{4}) denote the matrix in H, which has this vector in its first column. x_{3}^{2} + x_{4}^{2} is constant on the coset (x_{1}, x_{2}, x_{3}, x_{4})H_{E}. For ξ ∈ (x_{1}, x_{2}, x_{3}, x_{4}) · E^{⊥}, ξ ≠ 0, it holds:

x_{3}^{2} + x_{4}^{2} = \frac{ξ_{1}^{2} + ξ_{2}^{2}}{ξ_{1}^{2} + ξ_{2}^{2} + ξ_{3}^{2} + ξ_{4}^{2}} .

Now take φ ∈ S(E) such that

φ̃( (x_{1}, x_{2}, x_{3}, x_{4})H_{E}, ξ) = (x_{3}^{2} + x_{4}^{2})g(ξ) = \frac{(ξ_{1}^{2} + ξ_{2}^{2})g(ξ)}{ξ_{1}^{2} + ξ_{2}^{2} + ξ_{3}^{2} + ξ_{4}^{2}} =: F (ξ)

for ξ ∈ (x_{1}, x_{2}, x_{3}, x_{4}) · E^{⊥}, ξ ≠ 0. Obviously, φ is not an element of S(V)^{∧}, because F cannot be continuously extended to 0.

Next, H is being enriched by the central matrices of the form

( \begin{matrix}cos t − sin t sin t cos t & 0 \\ 0 & cos t − sin t sin t cos t \end{matrix} ) .  (17)

We study the interesting case E = ⟨Z_{2}, Z_{3}⟩. The matrices

X_{1} = ( \begin{matrix}0 & 1 \\ −1 & 0 \end{matrix} ; 0 ) , X_{2} = ( 0 ; \begin{matrix}0 & 1 \\ −1 & 0 \end{matrix} ) ,

Y = ( \begin{matrix}0 & 1 0 0 1 \\ \begin{matrix}−1 & 0 \\ 0 & −1 \end{matrix} & 0 \end{matrix} ) , Z = ( \begin{matrix}0 & \begin{matrix}0 & 1 \\ −1 & 0 \end{matrix} \\ \begin{matrix}0 & 1 \\ −1 & 0 \end{matrix} & 0 \end{matrix} )

form a basis of the Lie algebra of H. Now K^{1}(𝔥 ×_{s} ℝ^{4}) is spanned by

P = X_{1}(Z_{3}^{2} + Z_{4}^{2}) + X_{2}(Z_{1}^{2} + Z_{2}^{2}) + Y (Z_{1}Z_{4} − Z_{2}Z_{3}) − Z(Z_{1}Z_{3} + Z_{2}Z_{4}) .

For w = w_{1}Z_{1} + w_{2}Z_{4} ∈ E^{⊥} \ {0} we have: 𝔥_{w} = ⟨w_{2}^{2}X_{1} + w_{1}^{2}X_{2} + w_{1}w_{2}Y ⟩. Therefore, every 2-plane h · E^{⊥} through w has the form exp[t(w_{2}^{2}X_{1} + w_{1}^{2}X_{2} + w_{1}w_{2}Y)] · E^{⊥}, t ∈ ℝ. Since w̄ := w_{1}Z_{2} − w_{2}Z_{3} is orthogonal to all these planes (𝔥w̄ = 𝔥_{w} and w̄ ⊥ E^{⊥}), the action does not have property (G). Nevertheless, we can still dispense with the moment conditions. The latter are satisfied if the functions F_{m} in (15) are polynomials. So we proceed as follows.

Let F ∈ C^{∞}(ℝ^{4} \ {0}) have the property that its restriction to every plane h · E^{⊥} is a homogeneous polynomial of degree m. We shall show that F is a homogeneous polynomial of degree m.

At first, we establish this for the hyperplane ℝ^{3} × {0} in ℝ^{4}.

Let (x_{1}, x_{2}, x_{3}, x_{4}, t) denote the matrix (x_{1}, x_{2}, x_{3}, x_{4}) multiplied by the matrix (17). Then (x_{1}, x_{2}, x_{3}, x_{4}, t) · E^{⊥} lies in ℝ^{3} × {0} iff

x_{3} sin t + x_{4} cos t = x_{1} cos t + x_{2} sin t = 0 .

In that case,

h · w = \frac{1}{cos t} ( \begin{matrix}−w_{1}x_{2} sin 2t + w_{2}x_{3} sin 2t \\ w_{1}x_{2} cos 2t − w_{2}x_{3} cos 2t \\ w_{1}x_{3} + w_{2}x_{2} \\ 0 \end{matrix} )

for cos t ≠ 0 and h · E^{⊥} = ⟨Z_{2}, Z_{3}⟩ for cos t = 0. We obtain all planes through (0, 0, 1) in ℝ^{3}. Since F, restricted to each of them, is a homogeneous polynomial, we can write

F (ξ_{1}, ξ_{2}, ξ_{3}, 0) = A_{0}(ξ_{1}, ξ_{2}) + A_{1}(ξ_{1}, ξ_{2})ξ_{3} + … + A_{m}(ξ_{1}, ξ_{2})ξ_{3}^{m} .

If we write down this equation for m + 1 different values of ξ_{3} ≠ 0, we obtain a linear system in A_{0}, …, A_{m}. By Cramer’s rule we deduce that the A_{i}’s are C^{∞}-functions and, therefore, so is F |_{ℝ^{3}×{0}}. Now since F |_{ℝ^{3}×{0}} is homogeneous, it is a polynomial. We denote it by P_{0}.

If F (ξ_{1}, ξ_{2}, ξ_{3}, ξ_{4}) ≢ P_{0}(ξ_{1}, ξ_{2}, ξ_{3}), let 0 < k_{0} ≤ m be the highest possible exponent, such that

F_{1}(ξ_{1}, ξ_{2}, ξ_{3}, ξ_{4}) := \frac{F (ξ_{1}, ξ_{2}, ξ_{3}, ξ_{4}) − P_{0}(ξ_{1}, ξ_{2}, ξ_{3})}{ξ_{4}^{k_{0}}}

is a C^{∞}-function on ℝ^{4} \ {0}. This function is a homogeneous polynomial of the smaller degree m − k_{0} on every plane h · E^{⊥} that is not contained in ℝ^{3} × {0}. This means that every expression of the form

(\frac{∂}{∂w_{1}} )^{m−k_{0}+1−j}(\frac{∂}{∂w_{2}} )^{j}F_{1}(h · w)

vanishes for h = (x_{1}, x_{2}, x_{3}, x_{4}, t), (x_{3} sin t+x_{4} cos t, x_{1} cos t+x_{2} sin t) ≠ (0, 0). By continuity, it vanishes for every h. Consequently, F_{1} is a homogeneous polynomial on every plane h · E^{⊥} ⊆ ℝ^{3} × {0} too.

We repeat the argument with F_{1} instead of F, define a polynomial P_{1}, a number k_{1} and a function F_{2} with which we continue in the same manner. After finitely many steps

(P_{0}, k_{0}) → (P_{1}, k_{1}) → … → (P_{n}, k_{n})

we obtain a function F_{n+1} ∈ C^{∞}(ℝ^{4} \ {0}), constant on every plane h · E^{⊥}. Since all planes h · E^{⊥} ∈ ℝ^{3} × {0} contain (0, 0, 1), F_{n+1} is constant on ℝ^{3} × {0}. Since every other plane h · E^{⊥} meets ℝ^{3} × {0}, it is globally constant. We obtain:

F (ξ_{1}, ξ_{2}, ξ_{3}, ξ_{4}) = F_{n+1}ξ_{4}^{∑_{i=0}^{n} k_{i}} + ∑_{j=0}^{n} P_{n−j}(ξ_{1}, ξ_{2}, ξ_{3})ξ_{4}^{∑_{i=0}^{n−j−1} k_{i}}

(∑_{i=0}^{n} k_{i} = m).

References

[1] Borho, W., and J.-L. Brylinski, J.-L., Differential Operators on Homogeneous Spaces. I: Irreducibility of the Associated Variety for Annihilators of Induced Modules, Invent. Math. 69 (1982), 437–476.

[2] Felix, R., Radon transforms associated with certain group actions, Preprint.

[3] Gel’fand, I. M., S. G. Gindikin, and M. I. Graev, Integral geometry in affine and projective spaces, J. Soviet Math. 18, No. 3 (1982), 39–164 (translated from Itogi Nauki i Tekhniki, Seriya Sovremennye Problemy Matematiki 16 (1980), 53–226).

[4] Gel’fand, I. M., and M. I. Graev, Line complexes in the space ℂ^{n}, Functional Ann. Appl. 2 (1968), 219–229.

[5] Glaeser, G., Fonctions composées differentiables, Ann. Math. 77 (1963), 193–209.

[6] Gonzalez, F. B., On the range of the Radon d-plane transform and its dual, Trans. Amer. Math. Soc. 327, (1991), 601–619.

[7] Grinberg, E. L., Euclidean Radon transforms: ranges and restrictions, Contemp. Math. 63 (1987), 109–133.

[8] Helgason, S., A duality for symmetric spaces with applications to group representations, II. Differential equations and eigenspace representations, Advan. Math. 22 (1976), 187–219.

[9] Helgason, S., “Groups and Geometric Analysis, Integral Geometry, Invariant Differential Operators, and Spherical Functions,” Academic Press, Inc., Orlando etc., 1984.

[10] John, F., The ultrahyperbolic differential equation with four independent variables, Duke Math. J. 4 (1938), 300–322.

[11] Kurusa, Á., A Characterization of the Radon Transform’s Range by a System of PDEs, J. Math. Anal. Appl. 161 (1991), 218–226.

[12] Radon, J., Über die Bestimmung von Funktionen durch ihre Integralwerte längs gewisser Mannigfaltigkeiten, Ber. Verh. Sächs. Akad. Wiss. Leipzig, Math-Nat. Kl. 69 (1917), 262–277, reprinted in: Helgason, S., “The Radon transform,” Birkhäuser, Boston-Basel-Stuttgart, 1980, 177–192.

[13] Richter, F., On the k-dimensional Radon transform of rapidly decreasing functions, Lecture Notes in Mathematics 1209, Springer-Verlag, Berlin, Heidelberg, New York, 1986, 243–258.

[14] Richter, F., On fundamental differential operators and the p-plane Radon transform, J. Glob. Anal. Geom. 8, (1990), 61–75.

[15] Symeonidis, E., “Das Problem des Bildes bei der allgemeinen d-Ebenen-Transformation,” Dissertation, Katholische Universität Eichstätt, 1995.

[16] Whitney, H., Analytic extensions of differentiable functions defined in closed sets, Trans. Amer. Math. Soc. 36 (1934), 63–89.

Mathematisch-Geographische Fakultät Katholische Universität Eichstätt D-85071 Eichstätt e.symeonidis@ku-eichstaett.de

Received July 9, 1997 and in final form June 5, 1998