**1. introduction and statement of results **

The research in this article is largely motivated by the following result concerning a functional inequality on the boundary of bounded domains in the Euclidean space , proved in [MTX] Corollary 3.1.

Theorem 1([MTX]) Let be a bounded domain with smooth boundary . Let and be the mean curvature and the second fundamental form of with respect to the outward normal respectively. If , then

for any smooth function on . Here , denote the gradient, the Laplacian on respectively, and is the volume form on . Moreover, equality in (1) holds for some if and only if for some constants . Here are the standard coordinate functions on .

When and is convex, it is known ([MTX]) that the functional on the left side of (1) represents the second variation along of the Wang-Yau quasi-local energy ( [WY1], [WY2] ) at the -surface , lying in the time-symmetric slice , in the Minkowski spacetime . Thus, (1) can be relativistically interpreted as the stability inequality of the Wang-Yau energy at . The general case of such a stability inequality is implied by results in [CWY], [WY2] for a closed, embedded, spacelike -surface in that projects to a convex -surface along some timelike direction.

In this article, adopting a Riemannian geometry point of view, we generalize Theorem 1 to hypersurfaces that are boundaries of bounded domains in a simply connected space form. More generally, we give an analogue of (1) on the boundary of compact Riemannian manifolds whose metrics are *static* (see Definition 3).

First, we fix some notations. Given a constant , let and denote an -dimensional hyperbolic space of constant sectional curvature and an -dimensional open hemisphere of constant sectional curvature respectively.

Theorem 2Suppose is one of , and . Let be the positive function on given by

where is the distance function from a fixed point on . When , is chosen to be the center of so that on . Given a bounded domain with smooth boundary , let and be the mean curvature and the second fundamental form of respectively. If , then for any smooth function on ,

Here or is the sectional curvature of . Moreover, equality in (3) holds if and only if is the restriction of a function

Here are arbitrary constants, is identified with

in the -dimensional Minkowski space and is identified with

in the -dimensional Euclidean space .

The standard metrics on , , are all examples of static metrics which admit a *positive* {static potential}. We recall the following definition from [Co]:

Definition 3([Co]) A Riemannian metric on a manifold is called {static} if the linearized scalar curvature map at has a nontrivial cokernel, i.e. if there exists a nontrivial function on such that

Here , and denote the Hessian, the Laplacian and the Ricci curvature of respectively.

On a connected of dimension , the space of functions satisfying (5) has dimension at most (cf. [Co] Corollary 2.4). When is static on , a nontrivial solution to (5) is called a *static potential* of .

It is known that a static metric necessarily has constant scalar curvature (cf. [Co] Proposition 2.3). Indeed, direct calculation shows that is static with a positive static potential if and only if the Lorentz warped product satisfies where is the scalar curvature of (cf. [Co] Proposition 2.7). This interpretation explains why static metrics have been widely studied in the field of mathematical relativity (see e.g. [BM], [A], [Co], [Ch], [BS], [CG], [MR] ).

Our next theorem generalizes Theorem 1 to the boundary of a compact Riemannian manifold whose metric is static.

In Theorem 4, the fact that is taken as a nonpositive lower bound of the Ricci curvature of is restricted by the method of our proof (cf. Remark 2). Thus, if has positive Ricci curvature, (6) is always a strict inequality. However, in this case, if in addition that is Einstein, then can be chosen to be positive and (6) is sharp (cf. Remark 3).

If the metric is not static, we also give an inequality similar to that in Theorem 4 but under more stringent assumptions on the boundary and the interior curvature (see Theorem 6).

**2. proof of Theorems 2 and 3 **

Theorem 1 was derived in [MTX] as an application of Reilly’s formula [R]. (A different generalization of Theorem 1 was given in [MW], again by making use of Reilly’s formula.) To prove Theorem 2 and 4, we make use of the following weighted Reilly’s formula, recently derived by Qiu and Xia in [QX] Theorem 1.1.

Proposition 5[QX] Let be an -dimensional, compact Riemannian manifold with boundary . Given two functions , on and a constant , one has

For readers’ convenience, we include a proof of (7) below.

**Proof**: Direct calculation gives

* The integral of can be written as
*

* It follows from (8), (9) and the Bochner formula that
*

Using the fact

and

and along . Now (7) follows from (12) and the fact

Remark 1Formula (7) reduces to Reilly’s formula ([R] Equation(14)) when and .

Motivated by equation (5) in Definition 3 of static metrics, we can rewrite formula (7) as

It is the second line in (13) that prompts one to apply Proposition 5 to domains in a static manifold.

**Proof**: } As , given any nontrivial on , there exists a unique solution to

* On the other hand, taking trace of (5) gives
*

* where is the scalar curvature of (which is a constant). Plug this , together with and in (7), using (5), (13) and (15), we have
*

* Since , , and , (16) implies
*

* It follows from (17) that
*

* Moreover, by (16), equality in (18) holds only if
*

* Condition (19) implies either or . In the later case, it follows from that , i.e. is Einstein. We also note that (21) in fact follows from (20). This is because, if (20) holds, then at ,
*

which implies (21) since . This proves Theorem 4.

Remark 2In the above proof, the assumption is essentially used in only one place, i.e. to ensure

The other use of in the construction of is not essential because, by another theorem of Reilly ([R] Theorem 4) , one can still solve (14) in the case of , provided is not isometric to .

Remark 3If , then

regardless of the sign of . Therefore, the above proof also shows that inequality (6) still holds if the assumption “ and ” is replaced by that is Einstein. In this case, equality holds if and only if is the boundary value of some function that satisfies on .

Theorem 2 now follows from Theorem 4 and Remark 3.

**Proof**: } Each positive function in (2) is a solution to (5) when , or . Hence, inequality (3) follows from (6) in Theorem 4 and Remark 3.

Suppose the equality in (3) holds from a nontrivial . By Theorem 4 and Remark 3, is the boundary value of a function on satisfying

Since the standard metric on , and is also Einstein, the static equation (5) is equivalent to

Therefore, is the restriction of a static potential of to . Theorem 2 now follows from the fact that the space of solutions to (5) on is spanned by

Remark 4By [C] (p. 192-194) (cf. [T] Theorem 2 for a related result), it is known that if possesses a function with , then is locally a warped product metric in the sense that there exists a Riemannian manifold such that can be locally expressed as where is a function on an interval . In fact, their argument (which is local) shows that can be expressed as a function of and satisfies the linear ODE , and that . Also, and are unique up to multiplicative constants. Once these have been fixed, is determined by an additive constant. For example, when , is locally a product metric .

**3. A similar inequality **

When the metric is not static, there is an inequality similar to that in Theorem 4 but under more stringent conditions on the boundary and the interior curvature.

For a compact Riemannian manifold with boundary , we say it is star-shaped with respect to an interior point if every point in can be joined by a minimal geodesic starting from .

**Proof**: By Hessian comparison, we have

This implies

By diagonalizing , we see that

and . This implies that, for any function on ,

The proof then proceeds as in Theorem 4.

If the equality case holds, then as in the argument of Theorem 4, we have , which implies has constant curvature as we assume its curvature .

*Acknowledgements.* After this article was submitted, we learned that an inequality that is analogous to (3) in Theorem 2 was established by Chen, Wang and Yau ([CWY2]) in the study of quasi-local energy for spacetimes with a cosmological constant. We want to thank Professors Po-Ning Chen and Mu-Tao Wang for helpful discussions concerning (3).