Sbornik: Mathematics
RUS  ENG    JOURNALS   PEOPLE   ORGANISATIONS   CONFERENCES   SEMINARS   VIDEO LIBRARY   PACKAGE AMSBIB  
General information
Latest issue
Forthcoming papers
Archive
Impact factor
Guidelines for authors
License agreement
Submit a manuscript

Search papers
Search references

RSS
Latest issue
Current issues
Archive issues
What is RSS



Mat. Sb.:
Year:
Volume:
Issue:
Page:
Find






Personal entry:
Login:
Password:
Save password
Enter
Forgotten password?
Register


Sbornik: Mathematics, 2024, Volume 215, Issue 10, Pages 1269–1320
DOI: https://doi.org/10.4213/sm10087e
(Mi sm10087)
 

This article is cited in 1 scientific paper (total in 2 paper)

Moduli of rank $2$ semistable sheaves on rational Fano threefolds of the main series

D. A. Vasil'eva, A. S. Tikhomirovb

a Laboratory of Algebraic Geometry and Its Applications, National Research University Higher School of Economics, Moscow, Russia
b Faculty of Mathematics, National Research University Higher School of Economics, Moscow, Russia
References:
Abstract: The moduli spaces of semistable coherent sheaves of rank $2$ on the projective space $\mathbb{P}^3$ and the following rational Fano manifolds of the main series are investigated: the three-dimensional quadric $X_2$, the intersection of two four-dimensional quadrics $X_4$ and the Fano manifold of degree five $X_5$. For the quadric $X_2$ the boundedness of the third Chern class $c_3$ of rank $2$ semistable objects in $\mathrm{D}^b(X_2)$, including sheaves, is proved. An explicit description is presented for all moduli spaces of semistable sheaves of rank $2$ on $X_2$, including reflexive ones, with the maximal third class $c_3\ge0$. These spaces turn out to be irreducible smooth rational manifolds in all cases, apart from the following two: $(c_1,c_2,c_3)=(0,2,2)$ or (0,4,8). The first example of a disconnected module space of semistable rank $2$ sheaves with fixed Chern classes on a smooth projective variety is found: this is the second exceptional case $(c_1,c_2,c_3)= (0,4,8)$ on the quadric $X_2$. Several new infinite series of rational components of the moduli spaces of semistable sheaves of rank $2$ on $\mathbb{P}^3$, $X_2$, $X_4$ and $X_5$ are constructed, as also is a new infinite series of irrational components on $X_4$. The boundedness of the class $c_3$ is proved for $c_1=0$ and any $c_2>0$ for stable reflexive sheaves of general type on the varieties $X_4$ and $X_5$.
Bibliography: 30 titles.
Keywords: stable sheaves of rank $2$, moduli spaces of sheaves, Fano manifolds.
Funding agency Grant number
HSE Basic Research Program
The research of D. A. Vasil'ev was funded in the framework of the Basis Research Program of HSE University.
Received: 19.02.2024 and 01.07.2024
Published: 12.01.2025
Bibliographic databases:
Document Type: Article
Language: English
Original paper language: Russian

§ 1. Introduction

1.1.

In 1980 Hartshorne, while investigating in [15] the spectra of stable reflexive coherent sheaves of rank $2$ on the projective space $\mathbb{P}^3$, proved the boundedness of the third Chern class $c_3$ of these sheaves for fixed first and second Chern classes $c_1$ and $c_2$. The sharp estimates he obtained for the class $c_3$ have the form

$$ \begin{equation} c_3\leqslant c^2_2-c_2+2 \quad\text{if } c_1=0 \quad\text{and}\quad c_3\leqslant c^2_2 \quad\text{if } c_1=-1 \end{equation} \tag{1.1} $$
(see [15], Theorem 8.2). In the same work the irreducibility, smoothness and rationality of the moduli spaces of such sheaves were proved for $c_1=-1$, an arbitrary $c_2>0$ and the maximal $c_3=c^2_2$. In 2018 Schmidt [25], while investigating the properties of tilt stability in the derived category $\mathrm{D}^b(\mathbb{P}^3)$, proved that the estimates (1.1) are true for all semistable sheaves of rank $2$ on $\mathbb{P}^3$, and gave an explicit description of their moduli space for $-1\leqslant c_1\leqslant0$, $c_2>0$ and the maximal $c_3$. As a consequence, he obtained that these spaces are irreducible smooth rational projective varieties. It is not difficult to see that the moduli spaces of reflexive sheaves described by Hartshorne are open subsets of these manifolds (see Theorem 1.4 below). We also note that quite recently, in his 2023 work [27] Schmidt generalized the above results to the case of sheaves on $\mathbb{P}^3$ of all ranks from 0 to 4.

In this paper we study the moduli spaces of semistable rank $2$ sheaves on rational three-dimensional Fano varieties of the main series. There are four such varieties, namely, the projective space $X_1=\mathbb{P}^3$, the three-dimensional quadric $X_2$, the complete intersection $X_4$ of two quadrics in the space $\mathbb{P}^5$, and the section $X_5$ of the Grassmannian $\mathrm{Gr}(2,5)$ embedded in $\mathbb{P}^9$ according to Plücker by a linear subspace $\mathbb{P}^6$ (see, for example, [1] or [16]). Here the subscript $i$ at the variety $X_i$ denotes its projective degree. The Chern classes of bundles $E$ on $X=X_i$ are determined by three integers $c_1$, $c_2$ and $c_3$ (see formulae (2.3) below), and the corresponding moduli spaces (Gieseker–Maruyama moduli schemes) of semistable bundles of rank $2$ on $X$ will be denoted by $M_X(2;c_1,c_2,c_3)$.

The first direction of research in this paper concerns the question of the boundedness of the third Chern class $c_3$ of semistable rank $2$ sheaves on $X$ with fixed $c_1\in\{-1,0\}$ and $c_2\geqslant0$ and of estimates for this Chern class $c_3$. Using the tilt stability technique in the derived category $\mathrm{D}^b(X)$ we give a complete answer to this question for the three-dimensional quadric $X_2$ in the following theorem (see statements (3.1)–(4.2) in Theorem 3.1).

Theorem 1.1. (i) Let $E$ be a semistable sheaf of rank $2$ with $c_1=-1$ on the quadric $X_2$. Then $c_2\geqslant0$, and $c_3\leqslant\frac12c_2^2$ if $c_2$ is even, and, respectively, $c_3\leqslant\frac12(c_2^2-1)$ if $c_2$ is odd.

(ii) Let $E$ be a semistable sheaf of rank $2$ on $X_2$ with $c_1(E)=0$. Then $c_2\geqslant0$, and $c_3\leqslant\frac 12c_2^2$ if $c_2$ is even, and, respectively, $c_3\leqslant\frac 12(c_2^2+1)$ if $c_2$ is odd.

These estimates are sharp for all $c_3\geqslant0$.

The proof of this theorem is based on the study of the relationship between tilt semistability and Bridgeland semistability in $\mathrm{D}^b(X_2)$. The key here is Schmidt’s important technical result (2014) on the description of the subcategory of $\mathrm{D}^b(X_2)$ generated by a torsion pair in the sense of Bridgeland; see Proposition 2.4.

Unfortunately, no analogues of this result are known to date for the varieties $X_4$ and $X_5$. Therefore, for these varieties it is not possible to use the same method to obtain sharp upper bounds for the class $c_3$ for all semistable sheaves of rank $2$ on $X_4$ and $X_5$. However, using the more traditional technique of considering the behaviour of stable sheaves under standard birational transformations $X_4\dashrightarrow X_1$ and $X_5\dashrightarrow X_2$, we give a partial answer to the question of the boundedness of $c_3$ for a sufficiently wide class of sheaves on $X_4$ and $X_5$.

Namely, we consider stable reflexive sheaves of rank $2$ with $c_1=0$, called sheaves of general type in this paper. These sheaves $E$ on $X_i$, $i=4,5$, are such that $E|_l\cong\mathcal{O}_{\mathbb{P}^1}^{\oplus2}$ for a general line $l\subset X_i$, and lines for which either $E|_l\cong\mathcal{O}_{\mathbb{P}^1}(a)\oplus \mathcal{O}_{\mathbb{P}^1}(-a)$ with $a\geqslant2$, or $E|_l$ is not locally free, form a subset of dimension $\leqslant0$ in the base of the family of lines on $X_i$ (see Definition 4.1). In Theorem 4.7 we give examples of infinite series of components of moduli spaces of semistable sheaves in which the general sheaf is a reflexive sheaf of general type. (Presumably, the property of being a sheaf of general type holds for all stable reflexive sheaves of rank $2$ with $c_1=0$, that is, perhaps, an analogue of the Grauert–Mülich theorem, which is known to be valid for stable reflexive sheaves of rank $2$ on $X_1$, holds for them.) For sheaves of general type we prove the following theorem (see Theorems 6.2 and 6.1).

Theorem 1.2. Let $E$ be a stable reflexive sheaf of rank $2$ of general type with Chern classes $c_1=0$, $c_2>0$ and $c_3$ on the variety $X_4$ or $X_5$. Then the following inequalities are true for the class $c_3$ of the sheaf $E$.

(i) On $X_4$: $c_3\leqslant c_2^2-c_2+2$.

(ii) On $X_5$: $c_3\leqslant\frac 29c_2^2$ if $c_2$ is even, and, respectively, $c_3\leqslant\frac 29c_2^2+\frac 12$ if $c_2$ is odd.

Whether these estimates are sharp is an open question.

1.2.

The second direction of research in this paper is the construction of new infinite series (with growing class $c_2$) of moduli components of semistable sheaves of rank $2$ on the varieties $X_1$, $X_2$, $X_4$ and $X_5$, including an explicit description of general1 sheaves in these components. The first two infinite series of such components on $X_1=\mathbb{P}^3$ were constructed by Hartshorne in [14] (1978) for vector bundles (this is a series of instanton components with $c_1=0$ and a parallel series of components with $c_1=-1$) and in [15] (1980) for reflexive bundles with $c_1=-1$ and maximal $c_3$, and with $c_1=0$ and maximal spectrum. In 1978 Barth and Hulek [6] built another infinite series of families of stable vector bundles of rank $2$ on $\mathbb{P}^3$, and in 1981 Ellingsrud and Strømme proved [12] that these families are open subsets of irreducible components of moduli spaces. In [29] (1985) and [30] (1987) Vedernikov constructed new infinite series of families of stable vector bundles of rank $2$ on $\mathbb{P}^3$, and in 1984 Rao [23] built a more general series of families, which included the Vedernikov series. Ein described these families independently in 1988 and proved that they are open subsets of irreducible components of moduli spaces. In 2019 Kytmanov, A. S. Tikhomirov and S. A. Tikhomirov [19] proved the rationality of significant part of components of these series.

A large number of new infinite series of components of moduli spaces of semistable sheaves of rank $2$ on $\mathbb{P}^3$ were found by Jardim, Markushevich and A. S. Tikhomirov [17] in 2017 and by Almeida, Jardim and A. S. Tikhomirov [2] in 2022, and general sheaves in these components were described. In the work [25] mentioned above Schmidt presented a full description of all schemes of stable sheaves of rank $2$ with maximal Chern class $c_3$, proving their projectivity, irreducibility for any admissible values of the Chern classes $c_1$ and $c_2$, rationality and, in almost all cases, smoothness. Note that the last property of smoothness is a rather unexpected phenomenon for projective Gieseker–Maruyama moduli schemes of semistable sheaves of rank $2$ on three-dimensional varieties.

As concerns the varieties $X_2$, $X_4$ and $X_5$, by now only one infinite series of moduli components of semistable sheaves of rank $2$ has been found on each of them. These are the series of components containing families of instanton bundles as open sets. Instanton bundles on $X_2$ were defined by Costa and Miró-Roig [10] in 2009, and on $X_4$, $X_5$ and other Fano varieties by Kuznetsov [18] in 2012 and Faenzi [13] in 2013. In [13] Faenzi proved that families of instanton bundles on $X_2$, $X_4$ and $X_5$ are indeed open subsets of irreducible components of moduli spaces, which are reduced at a general point and have the expected dimension. In recent years an extensive number of works were devoted to the study of instanton series of bundles, a survey of which can be found, for example, in [3] and [9].

In this paper we construct several new infinite series of irreducible rational components of moduli spaces of semistable sheaves of rank $2$ on the varieties $X_1$, $X_2$, $X_4$ and $X_5$. We describe general sheaves in these components and prove their reflexivity, and also find the dimensions of the components constructed. These results are proved in Theorems 4.24.6. They are collected in the following theorem.

Theorem 1.3. Let $X$ be one of the varieties $X_1$, $X_2$, $X_4$ and $X_5$, and let $\mathcal{O}_X(1)$ be the ample sheaf on $X$ such that $\mathrm{Pic}(X)= \mathbb{Z}[\mathcal{O}_X(1)]$. Consider a sheaf $E$ of rank $2$ on $X$ defined by a nontrivial extensions of the form

$$ \begin{equation} 0\to F_i\to E\to G_j\to0, \qquad 1\leqslant i\leqslant3, \quad 1\leqslant j\leqslant2, \end{equation} \tag{1.2} $$
where $F_1=\mathcal{O}_X(-n)^{\oplus2}$ and $F_2=F(-n)$, where $F$ is a rank $2$ sheaf of one of types (I)–(III) described in § 4.3, $G_1=\mathcal{O}_S(m)$, where $S\in|\mathcal{O}_X(k)|$, and the sheaves $F_3$ and $G_2$ are defined in the case of the quadric $X=X_2$, namely, ${F_3=\mathcal{S}(-n)}$, where $\mathcal{S}$ is the spinor bundle on $X_2$ with $\det\mathcal{S}=\mathcal{O}_X(1)$, and $G_2=\mathcal{I}_{\mathbb{P}^1,S}(m)$, where ${S\in|\mathcal{O}_X(1)|}$ and $\mathbb{P}^1$ is a line on the surface $S$. Let $M_X(v)$ be the Gieseker–Maruyama moduli scheme of semistable sheaves on $X$ with Chern character ${v=\mathrm{ch}(E)}$, determined from the triple (1.2), and let
$$ \begin{equation} M:=\{[E]\in M_X(v)\mid E \textit{ is a Gieseker-stable extension } (1.2)\}. \end{equation} \tag{1.3} $$
Then the following statements are true:

(1) for $X_1$, $X_2$, $X_4$ and $X_5$ in the case when $i=j=1$, $k\geqslant1$, $n=\lceil\frac k2\rceil$ and $m<-n$,

(2) for $X_1$, $X_4$ and $X_5$ in the case when $i=2$, $j=1$, $k\geqslant1$, $n=\lfloor\frac k2\rfloor$ and ${m<-n}$,

(3) for $X_2$ in any of the following cases:

the set $M$ is a smooth dense open subset of an irreducible component $\overline{M}$ of the moduli scheme $M_X(v)$. Moreover, $M$ is a fine moduli space, and reflexive sheaves form a dense open set in $M$. In addition, all components $\overline{M}$ of the moduli spaces for the infinite series (1), (2) and (3.1)–(3.3) are rational varieties for each $X_i$, $i=1,2,4,5$, except for $X=X_4$ in the series (2), in which each component is irrational. In all cases the dimensions of the components $\overline{M}$ are found as polynomials from $\mathbb{Q}[k,m,n]$ or $\mathbb{Q}[m]$, respectively.

1.3.

A significant part of this paper is devoted to the study of semistable sheaves of rank $2$ with maximal class $c_3$ on the quadric $X_2$. We show that for $c_1\in\{-1,0\}$ and all values of the class $c_2$, except for a few small values, every such sheaf is given by an extension of the form (1.2), that is, in the notation (1.3) we have the equality $M=\overline{M}$. In this case the construction from the proof of Theorem 1.3 allows for a significant refinement, giving a full description of all moduli spaces of semistable sheaves with maximal class $c_3$ on $X_2$. In the remaining cases of small values of $c_2$ and maximal $c_3\geqslant0$ it is also possible to obtain an explicit description of moduli spaces. (The only case of negative maximal $c_3$ — the case $(c_1,c_2,c_{3\max})=(0,1,-1)$ — is specified in Remark 3.1 in this paper.) These results, proved in Theorems 5.15.4, are collected in the following two theorems.

Theorem 1.4. Let $X=X_2$ be a quadric and $M_X(v)$ be the Gieseker–Maruyama moduli scheme of semistable sheaves $E$ of rank $2$ on $X$ with Chern classes $(c_1,c_2,c_3)$, where $c_1\in\{-1,0\}$, $c_2\geqslant0$, and $c_3=c_{3\max}\geqslant0$ is maximal for each $c_2$, and

$$ \begin{equation*} v=\operatorname{ch}(E)=\biggl(2,c_1H,\frac12(c_1^2-c_2)H^2,\frac12\biggl(c_{3\max}+ \frac23c_1^3-c_1c_2\biggr)[\mathrm{pt}]\biggr), \end{equation*} \notag $$
where $H=c_1(\mathcal{O}_X(1))$. Then the following statements hold.

(1.i) For $c_1=-1$, and an even $c_2=2p$, $p\geqslant2$, and $c_{3\max}= \frac12c_2^2$ the variety $M_X(v)$ is the Grassmannization of two-dimensional quotient spaces of the vector bundle of rank $\frac14(c_2+2)^2$ on the space $\mathbb{P}^4$ that is defined by the first formula in (4.15) for $n=1$ and $m=-p$. In this case $\dim M_X(v)=\frac12(c_2+2)^2$.

(1.ii) For $c_1=-1$, an odd $c_2=2p+1$, $p\geqslant1$, and $c_{3\max}= \frac12(c_2^2-1)$ the variety $M_X(v)$ is the Grassmannization of two-dimensional quotient spaces of the vector bundle of rank $\frac14(c_2+1)(c_2+3)$ on the Grassmannian $\mathbb{G}=\mathrm{Gr}(2,4)$ that is defined by the second formula in (4.15) for $m=-p$. In this case $\dim M_X(v)=\frac12(c_2+1)(c_2+3)$.

(1.iii) For $c_1=0$, an odd $c_2=2p+1$, $p\geqslant1$, and $c_{3\max}= \frac12(c_2^2+1)$ the variety $M_X(v)$ is a projectivization of the vector bundle of rank $\frac12(c_2+1)(c_2+3)$ on the space $\mathbb{P}^4$ that is defined by (4.38) for $n=1$ and $m=-p$. In this case $\dim M_X(v)=\frac12c_2^2+2c_2+\frac92$.

(1.iv) For $c_1=0$, an even $c_2=2p$, $p\geqslant3$, and $c_{3\max}= \frac12c_2^2$ the variety $M_X(v)$ is a projectivization of the vector bundles of rank $\frac12c_2^2+2c_2+1$ on the Grassmannian $\mathbb{G}$ that is defined by (4.54) for $m=1-p$. In this case $\dim M_X(v)=\frac12c_2^2+2c_2+4$.

(2) In all the above cases the scheme $M_X(v)$ is irreducible and is a smooth rational projective variety, all sheaves in $M_X(v)$ are stable, the general sheaf in $M_X(v)$ is reflexive, and $M_X(v)$ is a fine moduli space.

Theorem 1.5. Under the assumptions and notation of Theorem 1.4 the following statements are true.

(1) For $c_1=-1$, $c_2=1$ and $c_{3\max}=0$ the variety $M_X(v)$ is the point $[\mathcal{S}(-1)]$.

(2) For $c_1=c_2=c_{3\max}=0$, $M_X(v)$ is the point $[\mathcal{O}_X^{\oplus2}]$.

(3) For $c_1=-1$, $c_2=2$ and $c_{3\max}=2$, we have $M_X(v)\simeq G(2,5)$.

(4) For $c_1=0$, $c_2=2$ and $c_{3\max}=2$ the scheme $M_X(v)$ is irreducible, has dimension 9 and is not smooth.

(5) For $c_1=0$, $c_2=4$ and $c_{3\max}=8$ the scheme $M_X(v)= M_X(2;0,4,8)$ is the union of two irreducible components $M_1$ and $M_2$. These components are described as follows.

(5.i) $M_1$ is a smooth rational variety of dimension 20, which is the projectivization of a locally free sheaf of rank 17 on the Grassmannian $\mathbb{G}$. It is a fine moduli space, and all sheaves in $M_1$ are stable. Moreover, the scheme $M_X(v)$ is nonsingular along $M_1$.

(5.ii) The scheme $M_2$ is irreducible, has dimension 21, and the polystable sheaves form a closed subset of $M_2$ of dimension 12, in which the scheme $M_X(v)$ is not smooth.

We single out statement (iii) of Theorem 5.4 as a separate theorem because of its importance.

Theorem 1.6. For the quadric $X=X_2$ the scheme $M_X(2,0,4,8)$ is disconnected:

$$ \begin{equation*} M_X(2;0,4,8)=M_1\sqcup M_2, \end{equation*} \notag $$
and its irreducible components $M_1$ and $M_2$ are described in statements (5.i) and (5.ii) of Theorem 1.5.

This result gives the first example of a disconnected moduli scheme of semistable sheaves of rank $2$ on a smooth projective three-dimensional variety. In all the few cases known so far in which the problem of the connectedness of the moduli scheme $M_X(2;c_1,c_2,c_3)$ with fixed $c_1$, $c_2$ and $c_3$ was discussed, the union of all known components of the moduli scheme turns out to be connected. In particular, in [17], Theorems 25 and 27, the connectedness of the scheme $M_{\mathbb{P}^3}(2;0,2,0)$ was proved, as well as the connectedness of the union of the seven irreducible components of the scheme $M_{\mathbb{P}^3}(2;0,3,0)$ known by 2017, and in the same paper [17] (Proposition 24) for an arbitrary positive integer $n$ the connectedness of the union of some number, growing with $n$, of known components of $M_{\mathbb{P}^3}(2;0,n,0)$ was proved. In [2], Main Theorem 3, the connectedness of the scheme $M_{\mathbb{P}^3}(2;-1,2,m)$ was proved for all admissible positive values of $m$, namely, for $m=0,2,4$. In our opinion, one possible reason why the scheme $M_{X_2}(2,0,4,8)$ in Theorem 1.6 is disconnected can be the fact that, unlike $\mathbb{P}^3$, the quadric $X_2$ is not a toric variety.

1.4.

We proceed on to a brief summary of the contents of the paper. In § 2 we recall some technical means we need to work in derived categories of coherent sheaves on Fano varieties, which are used in what follows. In § 3 the key result of the paper is proved, namely, Theorem 3.1 on the boundedness of the third Chern class $c_3$ of semistable objects, including sheaves, in $\mathrm{D}^b(X_2)$, and their complete classification is carried out for the maximum values of $c_3\geqslant0$. In § 4 we construct new infinite series of components of moduli spaces of rank $2$ semistable sheaves on the varieties $X_1$, $X_2$, $X_4$ and $X_5$. As a special case of these series, in § 5 we describe the moduli spaces of rank $2$ semistable sheaves with the maximal third Chern class on the quadric $X_2$. At the end of the same section we prove the disconnectedness of the moduli space $M_{X_2}(2,0,4,8)$. Section 6 is dedicated to a proof of the boundedness of the class $c_3$ of semistable rank $2$ reflexive sheaves of general type with $c_1=0$ on the varieties $X_4$ and $X_5$.

Notation

Throughout, we use the following notation:

§ 2. Preliminaries

2.1.

This section contains the necessary definitions and results that we use below.

Let $X$ be one of the varieties $X_i$, $i=1,2,4,5$. The cohomology ring $H^*(X,\mathbb{Z})$ is generated by the classes of a hyperplane section $H\in H^2(X,\mathbb{Z})$, a line $L\in H^4(X,\mathbb{Z})$ (understood as a projective line in the space $\mathbb{P}^{2+i}\supset X_i=X$ for $i=1,2$, or $X_i\hookrightarrow\mathbb{P}^{1+i}$ for $i=4,5$) and a point $\{\mathrm{pt}\}\in H^6(X,\mathbb{Z})$ (for simplicity we will also denote the class of a point by 1). We have

$$ \begin{equation} H^2=iL, \qquad H\cdot L=1, \qquad H^3=i \quad\text{and}\quad X=X_i, \quad i=1,2,4,5. \end{equation} \tag{2.1} $$
The slope $\mu(E)$ of a coherent sheaf $E\in\operatorname{Coh}(X)$ is defined by
$$ \begin{equation*} \mu(E)=\frac{(H^2\cdot\operatorname{ch}_1(E))}{(H^3\cdot\operatorname{ch}_0(E))} \end{equation*} \notag $$
(in the case of division by 0 we set $\mu(E)= +\infty$). A coherent sheaf $E$ is called $\mu$-(semi)stable if for any proper subsheaf $0\ne F\hookrightarrow E$ the inequality $\mu(F)\mathrel{{<}(\leqslant)}\mu(E/F)$ holds.

Let $f,g\in \mathbb{R}[m]$ be polynomials. If $\deg(f) < \deg(g)$, then we set $f > g$. If $d = \deg(f) = \deg(g)$ and $a$ and $b$ are the leading coefficients in $f$ and $g$ respectively, then we put $f \mathrel{{<}(\leqslant)} g$ if $\frac{f(m)}{a} \mathrel{{<}(\leqslant)} \frac{g(m)}{b}$ for all $m \gg 0$. For an arbitrary sheaf $E\in\operatorname{Coh}(\mathbb{P}^3)$ we define the numbers $a_i(E)$ for $i \in \{0, 1, 2, 3\}$ in terms of the Hilbert polynomial $P(E, m):=\chi(E(m))=a_3(E)m^3+a_2(E) m^2+ a_1(E) m + a_0(E)$. In addition, we set $P_2(E,m):=a_3(E)m^2+a_2(E)m+a_1(E)$. The sheaf $E \in \operatorname{Coh}(\mathbb{P}^3)$ is called (Gieseker-)(semi)stable (respectively, (Gieseker-)2-(semi)stable) if for any proper subsheaf of it $0\ne F \hookrightarrow E$ the inequality $P(F, m) \mathrel{{<}(\leqslant)}P(E/F, m)$ holds (respectively, $P_2(F, m) \mathrel{{<}(\leqslant)} P_2(E/F, m)$). The stability, 2-stability and $\mu$-stability of a sheaf satisfy the implications:

Let us recall the concept of tilt stability. Let $\beta\in\mathbb{R}$. We define the twisted Chern character by $\operatorname{ch}^\beta=e^{-\beta H}\cdot\operatorname{ch}$. Note that for $\beta\in\mathbb{Z}$ and any $E\in\mathrm D^b(X)$ the equality $\operatorname{ch}^\beta(E)=\operatorname{ch}(E(-\beta))$ holds. Let us present explicit formulae for the components $\operatorname{ch}_i^\beta=\operatorname{ch}_i^\beta(E)$:
$$ \begin{equation} \begin{gathered} \, \notag \operatorname{ch}_0^\beta=r, \qquad \operatorname{ch}_1^\beta=\operatorname{ch}_1-\beta H\operatorname{ch}_0, \qquad \operatorname{ch}_2^\beta=\operatorname{ch}_2-\beta H\operatorname{ch}_1+ \frac{\beta^2}{2} H^2\operatorname{ch}_0 \\ \text{and}\quad\operatorname{ch}_3^\beta=\operatorname{ch}_3-\beta H\operatorname{ch}_2+\frac{\beta^2}{2} H^2\operatorname{ch}_1-\frac{\beta^3}{6} H^3\operatorname{ch}_0. \end{gathered} \end{equation} \tag{2.2} $$
In particular, for
$$ \begin{equation*} \begin{gathered} \, X=X_2\quad\text{and}\quad E\in\mathrm D^b(X), \\ \text{where }v=\operatorname{ch}(E)=(r,cH,dH^2,e[\mathrm{pt}]), \qquad r,c\in\mathbb{Z}, \quad d\in\frac{1}{2}\mathbb{Z}, \quad e\in\frac{1}{6}\mathbb{Z} \end{gathered} \end{equation*} \notag $$
and
$$ \begin{equation} c(E)=(1,c_1(E),c_2(E),c_3(E))=(1,c_1H,c_2[l],c_3[\mathrm{pt}]), \qquad c_1,c_2,c_3\in\mathbb{Z}, \end{equation} \tag{2.3} $$
we have:
$$ \begin{equation} \nonumber \operatorname{ch}_0^\beta(E)=r, \qquad \operatorname{ch}_1^\beta(E)=(c-\beta r)H, \qquad \operatorname{ch}_2^\beta(E)=\biggl(d-\beta c+\frac{\beta^2}{2} r\biggr)H^2, \end{equation} \notag $$
$$ \begin{equation} \operatorname{ch}_3^\beta(E)=e-2\beta d+\beta^2c-\frac{\beta^3}{3}r, \end{equation} \tag{2.4} $$
$$ \begin{equation} c_1=c, \qquad c_2=c^2-2d, \qquad c_3=2e+\frac13c^3-2cd \end{equation} \tag{2.5} $$
and
$$ \begin{equation} \operatorname{ch}(E)=\biggl(r,c_1H,\frac12(c_1^2-c_2)H^2,\frac12\biggl(c_3+\frac23c_1^3-c_1c_2\biggr) [\mathrm{pt}]\biggr). \end{equation} \tag{2.6} $$
We consider a torsion pair
$$ \begin{equation*} \begin{gathered} \, \mathcal T_\beta=\{E\in\operatorname{Coh}(X)\colon\text{each quotient }E\to G \text{ satisfies }\mu(G)>\beta\}, \\ \mathcal F_\beta=\{E\in\operatorname{Coh}(X)\colon\text{each subsheaf }0\ne F\to E\text{ satisfies }\mu(F)\leqslant\beta\} \end{gathered} \end{equation*} \notag $$
and define the category $\operatorname{Coh}^\beta(X)$ as the subcategory $\langle\mathcal{F}_\beta[1], \mathcal{T}_\beta\rangle$ of $\mathrm{D}^b(X)$. For $\alpha\in\mathbb{R}_+$ the tilt-slope of an object $E\in\operatorname{Coh}^\beta(X)$ is defined by
$$ \begin{equation*} \nu_{\alpha,\beta}(E)=\nu_{\alpha,\beta}(\operatorname{ch}_0(E), \operatorname{ch}_1(E),\operatorname{ch}_2(E))=\frac{H\cdot \operatorname{ch}_2^\beta(E)-(\alpha^2/2)H^3\cdot\operatorname{ch}_0^\beta(E)} {H^2\cdot\operatorname{ch}_1^\beta(E)}. \end{equation*} \notag $$
An object $E\in\operatorname{Coh}^\beta(X)$ is called to be tilt-(semi)stable (or $\nu_{\alpha,\beta}$-(semi)stable) if for any subobject $0\ne F\hookrightarrow E$ we have $\nu_{\alpha,\beta}(F)\mathrel{{<}(\leqslant)}\nu_{\alpha,\beta}(E/F)$. In the case when the last inequality becomes equality we also say that $E$ is destabilized by the exact triple $0\to F\to E\to E/F\to0$.

Proposition 2.1. (i) An object $E\in\operatorname{Coh}^{\beta}(X)$ is $\nu_{\alpha,\beta}$-(semi)stable for $\beta<\mu(E)$ and $\alpha\gg0$ if and only if $E$ is a $2$-(semi)stable sheaf.

(ii) Let $X=X_2$. For any $\nu_{\alpha,\beta}$-semistable object $E\in\operatorname{Coh}^\beta(X_2)$ such that $\operatorname{ch}(E)=(r,cH,dH^2,e)$ the following inequalities hold:

$$ \begin{equation*} \Delta(E)=\Delta(\operatorname{ch} E):=\frac{(H^2\cdot\operatorname{ch}_1^\beta(E))^2-2(H^3 \cdot \operatorname{ch}_0^\beta(E)) (H\cdot\operatorname{ch}_2^\beta(E))}{(H^3)^2}=c^2-2rd\geqslant 0 \end{equation*} \notag $$
and
$$ \begin{equation*} \begin{aligned} \, W_{\alpha,\beta}(E) &:=\alpha^2\Delta(E)+\frac{4(H\cdot\operatorname{ch}_2^\beta(E))^2}{(H^3)^2} -\frac{6(H^2\cdot\operatorname{ch}_1^\beta(E))\operatorname{ch}_3^\beta(E)}{(H^3)^2} \\ &=(\alpha^2+\beta^2)(c^2-2rd)+(3re-2cd)\beta+4d^2-3ce\geqslant 0. \end{aligned} \end{equation*} \notag $$

Proof. Statement (i) was proved in [8], Proposition 14.2, for K3-surfaces, but the analogous proof holds in our case. For the proof of the first inequality in (ii), see Theorem 2.1 in [21], and for the second inequality, see Assumption C in [21].

The set of semistable objects changes with variation of pairs $(\alpha,\beta)$. Consider $H\cdot\operatorname{ch}_{\leqslant 2}$ as a mapping to $\Lambda=\mathbb Z^2\oplus\frac{1}{2}\mathbb{Z}$. A numerical wall with respect to a vector $v\in\Lambda$ is a nonempty proper subset $W$ of the upper half-plane $\{(\alpha,\beta)\mid \alpha>0\}$ given by an equation of the form $\nu_{\alpha,\beta}(v)=\nu_{\alpha,\beta}(w)$ with respect to some other vector $w\in\Lambda$. A subset $\mathcal{U}$ of the numerical wall $W$ is called an actual wall (or simply a wall) if the set of semistable objects $E\in \operatorname{Coh}^\beta(X)$ with $\operatorname{ch}(E)=v$ changes after crossing $\mathcal{U}$.

Proposition 2.2 ([25], Theorem 2.9). Let $v\in\Lambda$ be a fixed class. In what follows numerical walls are considered with respect to $v$.

(i) Numerical walls are either semicircles with centres on the $\beta$-axis, or rays parallel to the $\alpha$-axis. If $v_0\neq 0$, then there is a unique vertical numerical wall with the equation $\beta = v_1/v_0$. If $v_0 = 0$, then there are no actual vertical walls.

(ii) The curve $\nu_{\alpha, \beta}(v) = 0$ is a hyperbola that can degenerate for $v_0 = 0$ or $\Delta(v) = 0$. Moreover, this hyperbola crosses all semicircular walls at their highest points.

(iii) If two numerical walls defined by classes $w,u \in \Lambda$ intersect, then $v$, $w$ and $u$ are linearly dependent. In particular, these two walls coincide.

(iv) If a numerical wall has at least one point at which it is an actual wall, then it is an actual wall at every point.

(v) If $v_0\neq0$, then there is a largest semicircular wall on both sides of a single vertical wall.

(vi) If there is an actual wall defined numerically by an exact triple of tilt-semistable objects $0\to F\to E\to G\to0$ such that $\operatorname{ch}_{\leqslant 2}(E)= v$, then $\Delta(F) + \Delta(G) \leqslant \Delta(E)$. Moreover, equality holds if and only if $\operatorname{ch}_{\leqslant 2}(G)=0$.

(vii) If $\Delta(E) = 0$, then $E$ can destabilize only on a single numerical vertical wall. In particular, line bundles and their shifts by one are tilt-semistable everywhere.

2.2.

Throughout this section we consider the case when

$$ \begin{equation*} X=X_2 \end{equation*} \notag $$
is a three-dimensional quadric. The following lemma can be verified by direct calculation.

Lemma 2.1. Let $E\in\mathrm D^b(X)$ and $\operatorname{ch}(E)=(r,cH,dH^2,e)$. Then the equation $W_{\alpha,\beta}(E)=0$ is equivalent to $\nu_{\alpha,\beta}(r, cH,dH^2)=\nu_{\alpha,\beta}(2c,4dH,3eH^2)$. In particular, the equation $W_{\alpha,\beta}(E)=0$ describes a numerical wall in tilt stability.

Let us denote the radius of the semicircular wall $W_{\alpha,\beta}(E)=0$ by $\rho_W(E)$ and its centre by $s_W(E)$. The centre of the semicircular wall $\nu_{\alpha,\beta}(E)=\nu_{\alpha, \beta}(F)$ will also be denoted by $s(E,F)$.

Below we use the following results on tilt-semistable objects.

Proposition 2.3 ([21], Lemma 2.4). Suppose that an object $E$ is tilt-semistable and is destabilized by either a subobject $F\hookrightarrow E$ or a quotient $E\twoheadrightarrow F$ in $\operatorname{Coh}^\beta(X)$, inducing a nonempty semicircular wall $W$. Also assume that $\operatorname{ch}_0(F)>\operatorname{ch}_0(E)\geqslant0$. Then the radius $\rho_W$ of the wall $W$ satisfies the inequality

$$ \begin{equation*} \rho_W\leqslant\frac{\Delta(E)}{4\operatorname{ch}_0(F)(\operatorname{ch}_0(F)-\operatorname{ch}_0(E))}. \end{equation*} \notag $$

Recall the construction of the Bridgeland stability conditions on $X$. Let

$$ \begin{equation*} \mathcal T'_{\alpha,\beta} =\{E\in\operatorname{Coh}^\beta(X)\mid \text{each quotient }E\twoheadrightarrow G\ \text{satisfies}\ \nu_{\alpha,\beta}(G)>0\} \end{equation*} \notag $$
and
$$ \begin{equation*} \mathcal F'_{\alpha,\beta} =\{E\in\operatorname{Coh}^\beta(X)\mid \text{each subobject}\ 0\ne F\hookrightarrow E\ \text{satisfies } \nu_{\alpha,\beta}(F)\leqslant 0\}, \end{equation*} \notag $$
and set $\mathcal{A}^{\alpha,\beta}(X)=\langle \mathcal F'_{\alpha,\beta}[1],\mathcal T'_{\alpha,\beta}\rangle$. For any $s>0$ we define
$$ \begin{equation*} \lambda_{\alpha,\beta,s}=\frac{\operatorname{ch}^{\beta}_3 - s\alpha^2 H^2\cdot\operatorname{ch}^{\beta}_1}{H\cdot\operatorname{ch}^{\beta}_2 - ({\alpha^2}/{2})H^3\cdot \operatorname{ch}^{\beta}_0}. \end{equation*} \notag $$
An object $E\in\mathcal{A}^{\alpha, \beta}(X)$ is said to be $\lambda_{\alpha, \beta,s}$-(semi)stable if for each nontrivial subobject $F\hookrightarrow E$ we have $\lambda_{\alpha,\beta,s}(F)\mathrel{{<}(\leqslant)}\lambda_{\alpha,\beta,s}(E)$.

The derived category $\mathrm{D}^b(X)$ has a full strong exceptional collection $(\mathcal{O}_X(-1),\mathcal{S}(-1),\mathcal{O}_X,\mathcal{O}_X(1))$, where $\mathcal{S}$ is a spinor bundle on $X$.

Proposition 2.4 (see [24] and [26], Theorem 6.1, (2)). (i) Let $\alpha\!<\!1/3$, ${\beta\!\in\! [-1/2,0]}$ and $s=1/6$. For any $\gamma\in\mathbb{R}$ consider the torsion pair

$$ \begin{equation*} \mathcal T''_{\gamma} =\{E\in\mathcal A^{\alpha,\beta}(X)\mid \textit{each quotient } E\twoheadrightarrow G\textit{ satisfies }\lambda_{\alpha,\beta, s}(G)>\gamma \} \end{equation*} \notag $$
and
$$ \begin{equation*} \mathcal F''_{\gamma} = \{E \in \mathcal A^{\alpha, \beta}(X)\mid \textit{each subobject } 0\ne F\hookrightarrow E\textit{ satisfies } \lambda_{\alpha,\beta, s}(F)\leqslant\gamma\}. \end{equation*} \notag $$
Then there is $\gamma\in\mathbb{R}$ such that
$$ \begin{equation*} \langle \mathcal T''_{\gamma},\mathcal F''_{\gamma}[1]\rangle =\mathfrak C:=\langle\mathcal O_X(-1)[3],\mathcal S(-1)[2],\mathcal O_X[1],\mathcal O_X(1)\rangle. \end{equation*} \notag $$

(ii) Let the Chern character $v$ of an object from $\mathrm D^b(X)$ and $\alpha_0>0$, $\beta_0\in\mathbb{R}$ and $s>0$ be such that $\nu_{\alpha_0,\beta_0}(v)=0$, $H^2\cdot v_1^{\beta_0}>0$ and $\Delta(v)\geqslant0$. Assume that all $\nu_{\alpha_0, \beta_0}$-semistable objects of the class $v$ are $\nu_{\alpha_0,\beta_0 }$-stable. Then there is a neighbourhood $U$ of the point $(\alpha_0,\beta_0)$ such that for all $(\alpha,\beta)\in U$ satisfying $\nu_{\alpha,\beta}(v) >0$ an object $E\in\operatorname{Coh}^\beta(X)$ with $\operatorname{ch}(E)=v$ is $\nu_{\alpha,\beta}$-semistable if and only if it is $\lambda_{\alpha,\beta,s}$-semistable.

Remark 2.1. It follows from the proof of Proposition 3.2 in [21] that for a tilt-semistable object $E\in\operatorname{Coh}^\beta(X)$ with $\operatorname{ch}(E)=(1,0,-dH^2,e)$ the inequality $e\leqslant d(d+1)$ holds.

In what follows we use the equalities

$$ \begin{equation} \operatorname{ch}(\mathcal O_X(n))=\biggl(1,nH,\frac {n^2}2H^2,\frac{n^3}3[\mathrm{pt}]\biggr)\quad\text{and} \quad \operatorname{ch}(\mathcal S(-1))=\biggl(2,-H,0,\frac16[\mathrm{pt}]\biggr), \end{equation} \tag{2.7} $$
the exact triple of sheaves on $X$ (see [24])
$$ \begin{equation} 0\to\mathcal S(-1)\to V\otimes\mathcal O_X\xrightarrow{\varepsilon}\mathcal S\to 0, \qquad V\cong\mathbf k^4, \end{equation} \tag{2.8} $$
and the following properties of the spinor bundle $\mathcal{S}$ on $X$. In the triple (2.8) the epimorphism $\varepsilon$ gives an isomorphism of the spaces of global sections $h^0(\varepsilon)\colon V\xrightarrow{\cong}H^0(\mathcal{S})$. Let $B$ be the base of the family of lines on $X$. Throughout, we identify $B$ with $\mathbb{P}^3$ through the isomorphism $f\colon \mathbb{P}^3=\mathbb{P}(V) \xrightarrow{\cong}B$, $\mathbf{k} v\mapsto l$, where the line $l$ is the scheme of zeros of the section $h^0(\varepsilon)(v)$. Let $\mathbb{P}^4=\langle X\rangle$ be the projective hull of the quadric $X$, $\check{\mathbb{P}}^4=|\mathcal{O}_X(1)|$, and let $\mathbb{S}\subset\check{\mathbb{P}}^4\times X$ be the universal family of two-dimensional quadrics (hyperplane sections) on $X$. Let $\mathbb{G}:=\mathrm{Gr}(2,V)$ be the Grassmannian of lines in $\mathbb{P}^3$, and let $l_x$ be the line in $\mathbb{P}^3$ corresponding to a point $x\in\mathbb{G}$. Then $f(l_x)$ is a series of lines on the two-dimensional quadric $S_x\in\check{\mathbb{P}}^4$. It is not difficult to see that $\mu\colon \mathbb{G}\to\check{\mathbb{P}}^4$, $x\mapsto S_x$, is a double covering ramified in the dual quadric $\check{X}\subset \check{\mathbb{P}}^4$. The covering $\mu\colon \mathbb{G}\to\check{\mathbb{P}}^4$ defines a family of two-dimensional quadrics $\widetilde{\mathbb{S}}:=\mathbb{G} \times_{\check{\mathbb{P}}^4}\mathbb{S}\subset\mathbb{G}\times X$ with base $\mathbb{G}$ on $X$. The tautological subbundle $g\colon\mathcal{K}\to V\otimes\mathcal{O}_{\mathbb{G}}$ of rank $2$ on $\mathbb{G}$ and the epimorphism $\varepsilon$ in (2.8) define the composition $e\colon \mathcal{K}\boxtimes\mathcal{O}_X \xrightarrow{g\boxtimes\mathrm{id}_X}V \otimes\mathcal{O}_{\mathbb{G}} \boxtimes\mathcal{O}_X \xrightarrow{\mathrm{id}_{\mathbb{G}} \boxtimes\varepsilon}\mathcal{O}_{\mathbb{G}} \boxtimes\mathcal{S}$, and let $\mathbb{I}:=\operatorname{im}(e)\otimes\mathcal{O}_{\mathbb{G}} \boxtimes\mathcal{O}_X(-1)$. It is not difficult to see that $\ker(e)\cong\mathcal{O}_{\mathbb{G}}(-1)\boxtimes\mathcal{S}(-1)$. Thus, we have the following exact triple on $X\times\mathbb{G}$:
$$ \begin{equation} 0\to\mathcal O_{\mathbb{G}}(-1)\boxtimes\mathcal S(-1)\to\mathcal K\to\mathbb{I}\otimes \mathcal O_{\mathbb{G}}\boxtimes\mathcal O_X(1)\to0. \end{equation} \tag{2.9} $$
For an arbitrary point $x\in\mathbb{G}$ the restriction of this triple to $\{x\}\times X$ gives rise to an exact triple:
$$ \begin{equation} 0\to\mathcal S(-1)\to\mathcal O^{\oplus2}_X\to\mathcal I(1)\to0, \qquad \mathcal I:=\mathcal I_{\mathbb{P}^1,S}. \end{equation} \tag{2.10} $$
Here $\mathcal{I}_{\mathbb{P}^1,S}$ is the ideal sheaf of an arbitrary line $\mathbb{P}^1\subset S$ in the series of lines defined by $x$ on the two-dimensional quadric $S=S_x$.

§ 3. Semistable objects of rank $2$ in $\mathrm{D}^b(X_2)$ with maximal third Chern class $c_3$

3.1.

In this section we prove one of the main results of this paper, Theorem 3.1 on the description of rank $2$ semistable objects, including sheaves, from $\mathrm{D}^b(X_2)$ on the three-dimensional quadric

$$ \begin{equation*} X=X_2, \end{equation*} \notag $$
which have the maximal third Chern class $c_3$. Throughout what follows $Q_2$ denotes a two-dimensional quadric that is a hyperplane section of $X$, and $\mathbb{P}^1$ denotes a projective line on $Q_2$.

Theorem 3.1. Let $E\in\operatorname{Coh}^\beta(X)$ be a tilt-semistable object with $\operatorname{ch}(E)=(2,cH,dH^2,e)$, where $d\in\frac{1}{2}\mathbb{Z}$ and $e\in\frac{1}{6}\mathbb{Z}$. Then the following hold.

(1) If $c=-1$, then $d\leqslant 0$.

(2) If $c=0$, then $d\leqslant 0$.

(3) Let $E\!\in\!\operatorname{Coh}(X)$ be a semistable sheaf of rank $2$ with $c_1(E)\!=\!-H$, ${c_2(E)\!=\!c_2[l]}$ and $c_3(E)=c_3[\mathrm{pt}]$. Then $c_2\geqslant0$ and

(4) Let $E\in\operatorname{Coh}(X)$ be a semistable sheaf of rank $2$ with $c_1(E)=0$, $c_2(E)=c_2[l]$ and $c_3(E)=c_3[\mathrm{pt}]$. Then $c_2\geqslant0$ and

Remark 3.1. In statements (1.1)–(1.4), (2.1) and (2.3)–(2.6) of the theorem the maximum value $c_{3\max}(E)$ is nonnegative. Assertion (2.2) describes the only case when $c_{3\max}(E)=-1$ is negative. In this case it is not known whether an object $E$ with Chern classes $c_1(E)=0$, $c_2(E)=1$ and $c_3(E)=c_{3\max}(E)=-1$ is a Gieseker-(semi)stable sheaf.

3.2.

We preface the proof of Theorem 3.1 by a series of auxiliary statements, Lemmas 3.13.11.

Lemma 3.1. Let $E\in\operatorname{Coh}^\beta(X)$ be a tilt-semistable object with $\operatorname{ch}(E)=(2,0,0,e)$. Then $e\leqslant 0$, and if $e=0$, then $E\cong \mathcal{O}_X^{\oplus 2}$.

Proof. Since $E\in\operatorname{Coh}^\beta(X)$, we obtain $\beta< 0$. We have $W_{\alpha,\beta}(E)=6e\beta\geqslant 0$, so $e\leqslant 0$. If $e=0$, then, following the arguments in the proof of Proposition 4.1 in [26] and using Proposition 2.4, we find that $E\cong\mathcal O_X^{\oplus2}$. The lemma is proved.

Lemma 3.2. Let $E\!\in\!\operatorname{Coh}^\beta(X)$ be a tilt-semistable object with $\operatorname{ch}(E)\!=\!(2,-H,0,e)$. Then $e\leqslant\frac 16$, and if $e=\frac 16$, then $E\cong \mathcal{S}(-1)$.

Proof. Since $E\in\operatorname{Coh}^\beta(X)$, we have $\beta<-\frac 12$. The hyperbola $\nu_{\alpha,\beta}(E)=0$ intersects the $\beta$-axis at the point $\beta=-1$, so if there is an actual semicircular wall, it must intersect the line $\beta=-1$. In our case $H^2\cdot\operatorname{ch}_1^{-1}(E)$ has the smallest positive value possible, so if $E$ is $\nu_{\alpha,-1}$-semistable, it must be $\nu_{\alpha,-1}$-semistable for any $\alpha$: see Lemma 3.5 in [26]. Therefore, for $E$ there are no walls other than the unique vertical wall.

Note that $\nu_{1/4,(2-\sqrt{5})/4}(E(1))=0$, and therefore, applying Proposition 2.4 to $E(1)$, we obtain that $E(1)$ or $E(1)[1]$ belong to the category $\mathfrak C$. Solving the equation $\operatorname{ch}(E(1))=a\operatorname{ch}(\mathcal{O}_X(-1)[3]) +b\operatorname{ch}(\mathcal{S}(-1)[2])+c\operatorname{ch}(\mathcal{O}_X[1]) +d\operatorname{ch}(\mathcal{O}_X(1))$, where $a,b,c,d\in\mathbb Z$ are simultaneously nonnegative or nonpositive, we obtain the required bound for $\operatorname{ch}_3(E)$, and for its maximum value we find that $a=d=0$, $b=-1$ and $c=-4$. We obtain the exact triple $0\to \mathcal{S}(-2)\to\mathcal{O}_X(-1)^{\oplus 4}\to E\to 0$.

A calculation shows that the equality $\nu_{\alpha,\beta}(E)= \nu_{\alpha,\beta}(\mathcal{O}_X(-1))$ does not hold for any $(\alpha,\beta)$, and $\nu_{\alpha,\beta}(E)>\nu_{\alpha,\beta}(\mathcal{O}_X(-1))$. Therefore there is no morphism $E\twoheadrightarrow\mathcal{O}_X(-1)$. Since $\operatorname{Hom}(\mathcal{S}(-2),\mathcal{O}_X(-1))\cong\mathbb{C}^4$, any two injective morphisms $\mathcal{S}(-2)\to\mathcal{O}_X(-1)^{ \oplus 4}$ with tilt-semistable cokernel $E$ lie in the same orbit of the $\mathrm{GL}(4)$-action. So there is a unique object $E$ up to isomorphism, and the exact triple (2.8) shows that $E\cong \mathcal{S}(-1)$.

The lemma is proved.

Lemma 3.3. Let $E\in\operatorname{Coh}^\beta(X)$ be a tilt-semistable object with $\operatorname{ch}(E)=(2,0,-\frac 12H^2,e)$. Then $e\leqslant -\frac 12$.

Proof. Suppose that there is a semicircular wall intersecting the line $\beta=-1$. We have $\operatorname{ch}_{\leqslant 2}^{-1}(E)=(2,2H,\frac{H^2}{2})$, so in this case there is a destabilizing sheaf $F\hookrightarrow E$ with $\operatorname{ch}_{\leqslant 2}(F)=(R,H,xH^2)$. From the condition $\nu_{\alpha,-1}(F)= \nu_{\alpha,-1}(E)$ we obtain $x-\frac 14=\alpha^2(R-1)$. The case $R=1$ is impossible since $x\in\frac 12\mathbb Z$; for $R\geqslant 2$ we have $x>\frac 14$, and this contradicts the condition $\Delta(F)\geqslant 0$; if $R\leqslant 0$, then instead of $F$ we can work with the sheaf $E/F$.

Note that from the tilt semistability of $E$ it follows that $H^0(E)=0$. Also $H^2(E)\cong\operatorname{Hom}(E,\omega_X[1])=0$, since if a nonzero morphism $E\to\omega_X[1]$ existed, then we would have a semicircular wall intersecting the line $\beta=-1$. Todd’s class for $X$ has the form $\operatorname{td}(T_X)=(1,{3H}/{2},13H^2/{12},1)$. By the Riemann–Roch theorem we find that $e+\frac{1}{2}=\chi(E)=-h^1(E)-h^3(E)\leqslant 0$, that is, $e\leqslant-\frac 12$.

The lemma is proved.

Lemma 3.4. Let $E\in\operatorname{Coh}^\beta(X)$ be a tilt-semistable object with $\operatorname{ch}(E)=(2,0,-H^2,e)$. Then $e\leqslant 1$, and if $e=1$, then $E$ is destabilized by an exact triple $0\to\mathcal{O}_X(-1)^{\oplus 6}\to E\to\mathcal{S}(-2)^{\oplus 2}[1]\to 0$.

Proof. We have $\beta<0$, and each semicircular wall intersects the line $\beta=-1$. Since $\operatorname{ch}_{\leqslant 2}^{-1}(E)=(2,2H,0)$, for any destabilizing object $F\hookrightarrow E$ we have $\operatorname{ch}^{-1}(F)=(R,H,xH^2,e')$. Then the equality $\nu_{\alpha,-1}(F)=\nu_{\alpha,-1}(E)$ takes the form $x=(R-1)\alpha^2/2$. If $R=1$, then $x=0$. Since $\operatorname{ch}_1^{-1}(F)$ is minimal, the object $F$ is tilt-semistable for all $\alpha>0$ and $\beta<0$.

Since $F$ is stable, $H^0(F)=0$. If $H^2(F)\neq 0$, then by Serre duality there is a nontrivial morphism $F\to\mathcal O_X(-3)[1]$. However, such a morphism would define a semicircular wall, so $H^2(F)=0$.

From the Riemann–Roch theorem we find that $e'-\frac{1}{2}=\chi(F)=-h^1(F)-h^3(F)\leqslant 0$. Thus, $e'\leqslant\frac 12$. Since $\operatorname{ch}_{\leqslant 2}(E/F)=\operatorname{ch}_{\leqslant 2}(F)$, we also have ${\operatorname{ch}_3(E/F)\!\leqslant\!\frac 12}$. Therefore, in this case $e\leqslant 1$, and equality is attained for $e'=\frac 12$. Applying Proposition 2.4, in this case we obtain the exact triple ${0\to\mathcal{O}_X(-1)^{\oplus 3}\to F\,{\to}\, \mathcal{S}(-2)[1]\,{\to}\, 0}$ and a similar exact triple for $E/F$, from which the statement of the lemma follows for $R=1$.

If $R\neq 1$, then without loss of generality we can assume that $R\geqslant 2$; then $x>0$. However, the relation $\Delta(F)\geqslant 0$ implies that $x\leqslant\frac 14$, which is impossible since $x\in\frac 12\mathbb Z$. Finally, if $E$ has no destabilizing subobjects defining a semicircular wall, then we obtain the required statement by carrying out an argument for $E$ similar to the argument for $F$ in the case $R=1$.

The lemma is proved.

Lemma 3.5. Let $E\in\operatorname{Coh}^\beta(X)$ and $\operatorname{ch}(E)=(2,0,-\frac32 H^2,e)$. Then $e\leqslant \frac 52$, and if $e=\frac 52$, then $E$ is destabilized by an exact triple $0\to\mathcal{S}(-1)\to E\to\mathcal{O}_{Q_2}(-1)\to 0$.

Proof. We have $W_{0,-1}(E)=15-6e$, and if $e\geqslant\frac 52$, then $E$ destabilizes at $\beta=-1$, or the point ${\beta=-1}$, $\alpha=0$ is a limit point of a semicircular wall. By calculations we obtain $\operatorname{ch}^{-1}(E)=(2,2H, -\frac 12H,e-\frac 73)$, which means that there is a destabilizing subobject or a quotient $F$ with $\operatorname{ch}_1^{-1}(F)\in\{0,H\}$. If $\operatorname{ch}^{-1}(F)=(R,H,xH^2,e')$, then the relation $\nu_{\alpha,1}(E)=\nu_{\alpha,1}(F)$ is equivalent to the equality $x+\frac 14=\alpha^2({R-1})$. We can assume that $R>1$; then $\Delta(F)\geqslant 0$ implies that $x=0$. We have $\operatorname{ch}^{-1}(E/F)=(2-R,H,-\frac 12H^2,e'')$, and from the inequality $\Delta(E/F)\geqslant 0$ we obtain $R\in\{2,3\}$.

Let $R=2$. It follows from Lemma 3.2 that $e'\leqslant -\frac 16$, and in the case of equality $F\cong \mathcal{S}(-1)$. We have $\operatorname{ch}(E/F)=(0,H,-\frac 32H^2,e''+2)$, and each wall for $E/F$ intersects the line $\beta=-\frac 32$. Since $\operatorname{ch}_1^{-\frac 32}(E/F)=H$, for any destabilizing subobject $F'\hookrightarrow E/F$ we have $\operatorname{ch}_{\leqslant 2}^{-\frac 32}(F')=(r,\frac 12H,xH^2)$. From here we find that $0=2x-\alpha^2 r$, and therefore assuming that $r>0$ we can find the only possibility $r=1$ and $x=\frac 18$. From Lemma 3.1 it follows that $\operatorname{ch}_3(F')\leqslant -\frac 13$ (in the case of equality $F'\cong\mathcal{O}_Q(-1)$). In addition, $\operatorname{ch}_{\leqslant 2}((E/F)/F')=(-1,2H,-2H^2)$, so $\operatorname{ch}_3$ of this object is no greater than $\frac 83$. From these inequalities we find that $e\leqslant\frac 52$. When equality holds, we can assume that both the objects $F$ and $E/F$ are $\nu_{\alpha,\beta}$-semistable, which implies that $E/F\cong\mathcal{O}_{Q_2}(-1)$. In this case we have $\operatorname{Ext}^1(F,E/F)=0$, and therefore $F$ is indeed a subobject, rather than a quotient, of the object $E$. This vanishing is proved using the definition of an exceptional collection and the exact triple (2.8).

If $R=3$, then we denote the destabilizing subobject by $F'\hookrightarrow E$, and $\operatorname{ch}(F')= (3,-2H,\frac 12H^2,e')$. Any semicircular wall must intersect the line $\beta=-1$, but $\operatorname{ch}^{-1}_1(F')=H$. Hence $F'(1)[1]\in\mathfrak C$, and calculations give $e'\leqslant-\frac 16$. In addition, $\operatorname{ch}(E/F')=(-1,2H,-2H^2,e'')$, where $e''\leqslant \frac 83$. From these bounds we obtain $e\leqslant\frac 52$. In the case of equality we obtain the exact triples $0\to F'\to E\to \mathcal{O}_X(-2)[1]\to 0$ and $0\to \mathcal{O}_X(-1)^{ \oplus 5}\to F'\to \mathcal{S}(-2)[1]\to 0$. From the last exact triple it follows that $F'\cong\mathcal{O}_X(-1)\oplus F''$ and, as in the proof of Lemma 3.2, $\nu_{\alpha,\beta}(\mathcal{O}_X(-1))\neq \nu_{\alpha,\beta}(F'')$, so the object $F'$ is not tilt-semistable. We find that $e<\frac 52$ in the case of $R=3$.

Finally, if $\operatorname{ch}^{-1}_{\leqslant 2}(F)=(r,0,xH^2)$ and $\nu_{\alpha,\beta}(F)=\nu_{\ alpha,\beta}(E)$ on a semicircular wall with the limit point $\beta=-1$, $\alpha=0$, then the slope $\nu_{\alpha,\beta}(F)$ must tend to a finite limit as we approach this point along the wall, so $x=0$. From the inequality $0\leqslant\Delta(E/F)\leqslant\Delta(E)$ it follows that $0\leqslant r\leqslant 6$. The cases when $r\leqslant 5$ reduce to the case $R=2$ discussed above, while for $r=6$ we obtain $(E/F)(1)[1]\in\mathfrak C$, and calculations give $\operatorname{ch}_3((E/F)(1))\leqslant -\frac 56$, so that $\operatorname{ch}_3(E)\leqslant\frac32$.

The lemma is proved.

Lemma 3.6. Let $E\in\operatorname{Coh}^\beta(X)$ be a tilt-semistable object with $\operatorname{ch}(E)=(2,-H,-\frac 12H^2,e)$. Then $e\leqslant \frac 53$, and if $e=\frac 53$, then $E$ is destabilized by the an exact triple $0\to\mathcal{O}_X(-1)^{\oplus 3}\to E\to \mathcal{O}_X(-2)[1]\to 0$.

Proof. Assume that $e\geqslant\frac 53$. Then $W_{0,-\frac 32}(E)<0$, and therefore there is a wall intersecting the line $\beta=-\frac 32$. In addition, $\operatorname{ch}_{\leqslant 2}^{-\frac 32}(E)=(2,2H,\frac 14H^2)$, so there is a destabilizing subobject or a quotient $F$ of the object $E$ such that $\operatorname{ch}_1^{-\frac 32}(F)\in\{\frac 12H,H\}$. If $\operatorname{ch}_{\leqslant 2}^{-\frac 32}(F)=(R,\frac 12H,xH^2)$, then from the equality $\nu_{\alpha,- \frac 32}(E)=\nu_{\alpha,-\frac 32}(F)$ we obtain $2x-\frac 18=\alpha^2(2R-1)$. Note that $R$ must be odd since $H^2\cdot \operatorname{ch}^{-\frac 32}_1(F)$ is odd. Then the inequalities $\Delta(F)\geqslant 0,\Delta(E/F)\geqslant 0$ imply the $R\in\{-1,1\}$. If $R=-1$, then $x=-\frac 18$ and $\operatorname{ch}_3$ is maximized by the object $F\cong\mathcal{O}_X(-2)[1]$. If $R=1$, then $x=\frac 18$ and $F\cong\mathcal{O}_X(1)$. If $\operatorname{ch}_{\leqslant 2}^{-\frac 32}(F)=(R,H,xH^2)$, then $x-\frac 18=\alpha^2(R-1)$, $R$ is even and we can assume that $R>1$. But then $R=2$, $x=\frac 14$ and $F\cong\mathcal{O}_X(-1)^{\oplus 2}$. In any case we obtain the required inequality and an exact triple.

The lemma is proved.

Lemma 3.7. For $d\in\mathbb{Z}$ let $G_d\in\operatorname{Coh}^\beta(X)$ be a tilt-semistable object with $\operatorname{ch}(G_d)=(0,H,dH^2,e)$. Then the inequality $e\leqslant d^2-\frac 16$ holds, and in the case of equality $G_d\cong\mathcal{I}_{\mathbb{P}^1,Q_2}(d+1)$ for some two-dimensional quadric $Q_2\subset X$ and a line $\mathbb{P}^1\subset Q_2$.

Proof. The curve $\nu_{\alpha, \beta}(G_d)=0$ is a straight line $\beta=d$, and there are no semicircular walls, so $(G_d)(-d)[1]\in\mathfrak C$. Calculations yield the required inequality and an exact triple
$$ \begin{equation} 0\to \mathcal S(d-1)\to\mathcal O_X(d)^{\oplus 2}\to G_d\to 0. \end{equation} \tag{3.1} $$

As is known, the scheme of zeros $(s)_0$ of any section $0\ne s\in H^0(\mathcal{S})$ is a reduced projective line $\mathbb{P}^1\subset X$, from which we obtain an exact triple $0\to\mathcal{O}_X\xrightarrow{s}\mathcal{S}\to\mathcal{I}_{\mathbb{P}^1}(1)\to0$. Hence $\operatorname{coker}(\mathcal{O}_X^{\oplus 2} \xrightarrow{s,s'}\mathcal{S})=\operatorname{coker}(\mathcal{O}_X \xrightarrow{\phi}\mathcal{I}_{\mathbb{P}^1}(1)$. Consider the two-dimensional quadric $Q_2=\operatorname{supp}\operatorname{coker}(\phi)$. The morphism $\mathcal{I}_{Q_2}(1)\cong\mathcal{O}_X \xrightarrow{\phi}\mathcal{I}_{\mathbb{P}^1}(1)$ induces an exact triple $0\to\mathcal{I}_{Q_2}(1) \xrightarrow{\phi}\mathcal{I}_{\mathbb{P}^1}(1) \to\mathcal{I}_{\mathbb{P}^1,Q_2}(1)\to0$. This implies that the triple $0\to\mathcal{O}_X^{\oplus 2}\to\mathcal{S}\to\mathcal{I}_{\mathbb{P}^1,Q_2}(1)\to0$ is exact. Applying the functor ${R}\mathcal{H}om(\,\cdot\,,\mathcal{O}_X(d))$ to it and taking the easily verifiable isomorphism $\operatorname{Ext}^1(\mathcal{I}_{\mathbb{P}^1,Q_2}(1),\mathcal{O}_X(d)) \cong\mathcal{I}_{\mathbb{P}^1,Q_2}(d+1)$ into account we obtain an exact triple $0\to\mathcal{S}(d-1)\to\mathcal O(d)^{\oplus 2}\to\mathcal{I}_{\mathbb{P}^1,Q_2}(d+1)\to 0$. This triple is isomorphic by construction to the triple (3.1) for a suitable choice of the sections $s$ and $s'$, hence $G_d\cong\mathcal{I}_{\mathbb{P}^1,Q_2}(d+1)$.

The lemma is proved.

Lemma 3.8. Let $E\in\operatorname{Coh}^\beta(X)$ be a tilt-semistable object with $\operatorname{ch}(E)=(2,0,-2H^2,e)$. Then $e\leqslant 4$, and if $e=4$, then $E$ is destabilized by one of the exact triples

$$ \begin{equation} 0\to \mathcal O_X(-1)^{\oplus 4}\to E\to\mathcal O_X(-2)^{\oplus 2}[1]\to 0 \end{equation} \tag{3.2} $$
and
$$ \begin{equation} 0\to\mathcal S(-1)\to E\to \mathcal I_{\mathbb{P}^1,Q_2}(-1)\to 0. \end{equation} \tag{3.3} $$
In the case when the object $E$ is included in the exact triple (3.3), it is tilt-stable for $\beta<0$ and $\alpha\gg 0$ and not included in the exact triple (3.2).

Proof. We have $W_{0,-1}(E)=24-6e$, so the inequality $W_{0,-1}\geqslant 0$ is equivalent to $e\leqslant 4$. Therefore, it is sufficient to check that $e\leqslant 4$ for objects that have a semicircular wall at $\beta=-1$. We have $\operatorname{ch}_{\leqslant2}^{-1}(E)=(2,2H,-H^2)$, so in this case there is a destabilizing subobject or a quotient $F$ of the object $E$ with $\operatorname{ch}_1^{-1}(F)=H$. Let $\operatorname{ch}_{\leqslant2}^{-1}(F)=(R,H,xH^2)$. Then $\nu_{\alpha,-1}(F) =\nu_{\alpha,-1}(E)$ implies that $x+\frac 12=\frac{\alpha^2}{2}(R-1)$. We can assume that $R\geqslant2$, and then the inequality $\Delta(F)\geqslant 0$ implies that $x\in\{-\frac 12,0\}$.

Assume that $x=-\frac 12$. From the inequality $\Delta(E/F)\geqslant 0$ we obtain ${R\in\{2,3\}}$. If ${R=2}$, then $\operatorname{ch}_{\leqslant 2}(F)=(2,-H,-\frac 12H^2)$ and $\operatorname{ch}_{\leqslant2}(E/F)=(0 ,H,-\frac 32H^2)$ (we assume by abuse of notation that $F$ is a subobject of $E$). Therefore, using Lemma 3.6 and the proof of Lemma 3.5 we obtain the required inequality and one of the exact triples

$$ \begin{equation} 0\to F\to E\to\mathcal O_{Q_2}(-1)\to 0 \end{equation} \tag{3.4} $$
and
$$ \begin{equation} 0\to \mathcal O_{Q_2}(-1)\to E\to F\to 0, \end{equation} \tag{3.5} $$
where $F\in M(2,-H,-\frac 12H^2,\frac 53)$. By Lemma 3.6 we have an exact triple $0\to\mathcal{O}_X(-1)^{\oplus 3}\to F\to\mathcal{O}_X(-2)[1]\to 0$, and there is also an exact triple $0\to\mathcal{O}_X(-1)\to\mathcal{O}_{Q_2}(-1)\to\mathcal{O}_X(-2)[1]\to 0$. Considering long exact sequences of groups $\operatorname{Ext}$ we see that $\operatorname{Ext}^1(\mathcal{O}_X(-1),F) =\operatorname{Ext}^1(\mathcal{O}_X(-1),\mathcal{O}_{Q_2}(-1))=0$, and we obtain a morphism $F'=\mathcal{O}_X(-1)^{\oplus4}\cong \mathcal{O}_X(-1)^{\oplus 3}\oplus\mathcal{O}_X(-1)\to E$. By the Snake Lemma this morphism is injective and $E/F'\cong\mathcal{O}_X(-2)^{\oplus 2}[1]$.

If $R=3$, then $\operatorname{ch}_{\leqslant 2}(F)=(3,-2H,0)$ and $\operatorname{ch}_3(E/F)$ is maximized by the object $\mathcal{O}_X(-2)[1]$. The curve $\nu_{\alpha,\beta}(F)=0$ intersects the $\beta$-axis at the point $\beta=-\frac 43$, hence each semicircular wall for $F$ intersects the line $\beta= -\frac 43$. We have $\operatorname{ch}_{\leqslant2}^{-\frac 43}(F)=(3,2H,0)$, and therefore there is a destabilizing subobject or a quotient $G$ of the object $E$ with $\operatorname{ch}_1^{-\frac 43}(G)\in\{\frac 13H,\frac 23H,H\}$. In the first case we have $G\cong\mathcal{O}_X(-1)$, in the second case $G\cong\mathcal{O}_X(-1)^{ \oplus2}$ or $G\cong\mathcal{O}_X(-2)[1]$, and in the third $G\cong\mathcal{O}_X(-1)^{\oplus 3}$. In any case we obtain the required inequality and the exact triple (3.2).

Now assume that $x\mkern-2mu=0$. Then $R\mkern-2mu=2$, and we find that $\operatorname{ch}_{\leqslant 2}(F)\mkern-2mu=(2,-H,0)$ and $\operatorname{ch}_{\leqslant 2}(E/F)=(0,H,-2H^2)$. Lemmas 3.2 and 3.7 yield the required inequality and the exact triple (3.3).

We have proved that $e\leqslant 4$. For $e=4$ note that the exact triple (3.2) defines the wall $W_{\alpha,\beta}(E)=0$, so this wall is the smallest one, and repeating the arguments from the proof of Theorem 5.1 in [26] we show that any object destabilized by this wall is given by the exact triple (3.2). Any other wall intersects the line $\beta=-1$ at a point with $\alpha>0$, and therefore the above argument applies to them when $R\geqslant 2$. If $R=1$, then, because $W_{0,-1}(F)=6-3\operatorname{ch}_3(F)$ and $\operatorname{ch}_1^{-1}(F)$ is minimal, we find that $\operatorname{ch}_3(F)\leqslant 2$ and the wall $W_{\alpha,\beta}(F)=0$ is unique. This wall is given by the exact triple $0\to\mathcal{O}_X(-1)^{\oplus 2}\to F\to\mathcal{O}_X(-2)[1]\to 0$, from which, in combination with a similar exact triple for $E/F$, by the Snake Lemma we obtain a monomorphism $\mathcal{O}_X(-1)^{\oplus 4}\hookrightarrow E$ and the exact triple (3.2).

To prove the statement about the tilt stability of an object $E$ in (3.3) it is enough to check its tilt stability for $\beta=-1$ and $\alpha>1$. Because $\nu_{1,-1}(\mathcal{S}(-1))=\nu_{1,-1}(\mathcal{I}_{\mathbb{P}^1,Q_2}(-1))$, the object $E$ is $\nu_{1,-1}$-semistable. If $F$ is a subobject that destabilizes $E$ at $\beta=-1$, then $\operatorname{ch}_1^{-1}(F)=H$ (the case $\operatorname{ch}_1^{-1}(F)=0$ is impossible since then we have $\nu_{1,-1}(F)=+\infty$, which contradicts the $\nu_{1,-1}$-semistability of $E$, and in the case when $\operatorname{ch}_1^{-1}(F)=2H$ we have $\nu_{\alpha,-1}(F)<\nu_{\alpha,-1}(E/F)=+\infty$, so that $F$ does not destabilize $E$). Repeating the argument carried out above in the proof of this lemma, we obtain that either $F\cong\mathcal{S}(-1)$, or $E$ is included in an exact triple of the form (3.2). Direct calculation shows that $\mathcal{S}(-1)$ does not destabilize $E$ when $\beta=-1$ and $\alpha>1$, and for any object $E$ in (3.2) we have $\operatorname{Hom}(\mathcal{S}(-1),E) =\operatorname{Hom}(E,\mathcal{I}_{\mathbb{P}^1,Q_2}(-1))=0$, that is, $E$ cannot be included in both (3.2) and (3.3).

The lemma is proved.

Lemma 3.9. Let $E\in\operatorname{Coh}^\beta(X)$ be a tilt-semistable object with $\operatorname{ch}(E)=(2,-H,-H^2,e)$. Then $e\leqslant{19}/{6}$, and if $e={19}/{6}$, then $E$ is destabilized by an exact triple

$$ \begin{equation} 0\to\mathcal O_X(-1)^{\oplus 2}\to E\to \mathcal I_{\mathbb{P}^1, Q_2 }(-1)\to 0. \end{equation} \tag{3.6} $$

Proof. Suppose that $e\geqslant{19}/{6}$. Then $W_{0,-3/2}(E)<0$, and we have $\operatorname{ch}_{\leqslant 2}^{-3/2}(E)=(2,2H,-\frac 14H^2)$. Therefore, there is a destabilizing subobject or a quotient $F$ of $E$ such that $\operatorname{ch}_1^{-3/2}(F)\in \{\frac 12H,H\}$. Suppose that $\operatorname{ch}_{\leqslant 2}^{-3/2}(F)=(R,H,xH^2)$; then the equality $\nu_{\alpha,-3/2}(F)=\nu_{\alpha,-3/2}(E)$ implies that $x+\frac 18 =\alpha^2(R-1)$. Assume that $R>1$, then $-\frac 18<x\leqslant\frac 14$. If $x=0$, then $\Delta(E/F)\geqslant 0$, and since Chern classes are integers, it follows that $R=4$. Hence we obtain in the usual way that $e\leqslant\frac 76$. The case $x=\frac 18$ is impossible by numerical considerations, and if $x=\frac 14$, then $R=2$, and we obtain the required inequality and the exact triple (3.6).

Suppose that $\operatorname{ch}_{\leqslant 2}^{-3/2}(F)=(R,\frac 12H,xH^2)$. Then the equality $\nu_{\alpha,-3/2}(F)=\nu_{\alpha,-3/2}(E)$ implies that $2x+\frac 18=\alpha^2 (2R-1)$. For $R\geqslant 1$ there is only one possibility, $R=1$ and $x=\frac 18$, and then $\operatorname{ch}_3(F)$ is maximized by the sheaf $F\cong\mathcal O_X(-1)$, and we also obtain the exact triple (3.6). For $R<1$ there is only one possibility, $R=-1$ and $x=-\frac 18$, so that $F\cong\mathcal{O}_X(-2)[1]$. In this case we also obtain $e\leqslant\frac 76$.

The lemma is proved.

3.3.

Now we can proceed to the general case.

Lemma 3.10. Let $E\in\operatorname{Coh}^\beta(X)$ be a tilt-semistable object with $\operatorname{ch}(E)=(2,cH,dH^2,e)$. Assume that either

Then $E$ is destabilized along a semicircular wall by a subobject or a quotient of rank at most $2$.

Proof. (1) Suppose that $c=-1$, $d\leqslant -\frac 32$ and $e\geqslant d^2-2d+\frac 16$. Then calculations give
$$ \begin{equation*} \frac{\rho_W(E)}{\Delta(E)}\geqslant -\frac{9}{64}\, d-\frac{19}{256}-\frac{9}{128(1-4d)} -\frac{27}{64(1-4d)^2}+\frac{81}{256(1-4d)^3}. \end{equation*} \notag $$
Bounding each term, we find that $\rho_W(E)/\Delta(E)>\frac{1}{12}$, which means that we can use Proposition 2.3.

(2) Suppose that $c=0$, $d\leqslant -\frac 52$ and $e\geqslant d^2$. Then calculation gives

$$ \begin{equation*} \frac{\rho_W(E)}{\Delta(E)}\geqslant -\frac{9}{64}\, d-\frac{1}{4}, \end{equation*} \notag $$
and the application of Proposition 2.3 completes the proof.

Lemma 3.11. Let $E\in\operatorname{Coh}^\beta(X)$ be a tilt-semistable object with $\operatorname{ch}(E)=(2,cH,dH^2,e)$.

(1) Let $c=-1$ and $d\leqslant-\frac 32$. If $E$ is destabilized by a subobject $F$ of rank $1$, then $e\leqslant d^2-2d+\frac{5}{12}$. If $e=d^2-2d+\frac{5}{12}$ with noninteger $d$, or $e=d^2-2d+\frac 16$ with integer $d$, then $F\cong\mathcal{O}_X(-1)$ and $E$ is destabilized by a subobject of rank $2$.

(2) Let $c=0$ and $d\leqslant-\frac 52$. If $E$ is destabilized by a subobject or quotient $F$ of rank $1$, then $e<d^2$.

Proof. (1) Suppose that $c=-1$, $d\leqslant-\frac 32$ and $e\geqslant d^2-2d+\frac 16$. Calculation gives $W_{0,-3/2}(E)=4d^2-6d+\frac 94-6e\leqslant -2d^2+6d+\frac 54<0$, and therefore
$$ \begin{equation*} 0<H^2\cdot\operatorname{ch}_1^{-3/2}(F)<H^2\cdot\operatorname{ch}_1^{-3/2}(E)=2. \end{equation*} \notag $$

Hence $\operatorname{ch}_1(F)\in\{-H,0\}$. Assume that $\operatorname{ch}_1(F)=-H$. Let us show that $F$ is a subobject, rather than a quotient, of the object $E$. We have $\operatorname{ch}(F)= (1,0,-yH^2,z)\cdot\operatorname{ch}(\mathcal O_X(-1))$, and Remark 2.1 shows that $z\leqslant y(y+1)$. Calculation gives

$$ \begin{equation*} s_W(E)=\frac{d+3e}{4d-1}\leqslant\frac{6d^2-10d+1}{8d-2}\quad\text{and} \quad s(E,F)=d+2y-1. \end{equation*} \notag $$
Since $s(E,F)\leqslant s_W(E)$, we have
$$ \begin{equation*} y\leqslant\frac{d^2+1}{2-8d}<-\frac d2-\frac 14. \end{equation*} \notag $$

Using Remark 2.1 for the quotient $E/F$, we find that

$$ \begin{equation*} e\leqslant\biggl(\frac 12-d-y\biggr)\biggl(\frac 32-d-y\biggr)+y(y+1)+2y-\frac 13. \end{equation*} \notag $$

The right-hand side of this inequality is a quadratic function of $y$ with a minimum at the point $y=-d/2-1/4$. Therefore, its maximum is attained at the point $y=0$ for $F\cong\mathcal{O}_Q (-1)$, and we obtain $e\leqslant d^2-2d+{5}/{12}$. Note also that for $y>0$ we have $e\leqslant d^2-d+{17}/{12}<d^2-2d+1/6$, so for the integer $d$ we also find that $y=0$. Now from the proof of Proposition 3.2 in [21] it follows that there is a destabilizing morphism $\mathcal{O}_X(-1)\to G=E/F$, and the quotient has Chern character $\operatorname{ch}_{\leqslant 2}(G')=(0,H,(d-1)H^2)$. The argument in the proofs of Lemmas 3.5 and 3.8 shows that for noninteger $d$ the character $\operatorname{ch}_3(G')$ is maximized by the sheaf $G'\cong\mathcal{O}_{Q_2}(d-\frac 12)$, while for the integer $d$ there is an exact triple

$$ \begin{equation*} 0\to \mathcal S(d-2)\to\mathcal O_X(d-1)^{\oplus 2}\to G'\to 0. \end{equation*} \notag $$
Calculation shows that in any case $\operatorname{Ext}^1(F,G)=0$, so $F$ is indeed a subobject of $E$ and there is a destabilizing morphism $\mathcal{O}_X(-1)^{\oplus 2}\to E$.

(2) Suppose that $c=0$, $d\leqslant-\frac 52$ and $e\geqslant d^2$. Calculations show that $W_{0,-1}(E)=4d^2-4d-6e\leqslant-2d^2-4d<0$, and therefore

$$ \begin{equation*} 0<H^2\cdot\operatorname{ch}_1^{-1}(F)<H^2\cdot\operatorname{ch}_1^{-1}(E)=2. \end{equation*} \notag $$
This means that $\operatorname{ch}_1(F)=0$, but such $F$ do not define a semicircular wall.

The lemma is proved.

3.4.

Proof of Theorem 3.1. The inequalities for $d$ follow from the first inequality in Proposition 2.1, (ii). Other statements of the theorem are proved by induction on $\Delta(E)$, and the base of induction is given by Lemmas 3.13.9. Lemma 3.10 shows that $E$ is destabilized by a subobject $F\hookrightarrow E$, where $F$ has rank $R\in\{0,1,2\}$. Lemma 3.11 deals with the case $R=1$. Assume that $R=2$. In what follows we show that the case $R=0$ is only realized when $E$ is a direct sum.

(1) Suppose that $c=-1$, $d\leqslant-\frac 32$ and $e\geqslant d^2-2d+\frac 16$. As in the proof of Lemma 3.11, we have $W_{0,-3/2}(E)<0$, and also $W_{0,-2}(E)<0$. Then the inequalities $H^2\cdot\operatorname{ch}_1^{-3/2}>0$ and $H^2\cdot\operatorname{ch}_1^{-2}(F)<H^2\cdot\operatorname{ch}_1^{-2}(E)$ imply in combination that $\operatorname{ch}_1(F)=-2H$. Therefore, we can assume that $\operatorname{ch}(F)=(2,0,yH^2,z)\cdot\operatorname{ch}(\mathcal O_X(-1))$. If $y=0$, then $z\leqslant 0$, and the arguments in the proof of Lemma 3.11 give the required statement. Assume that $y\leqslant-\frac 12$. By the induction hypothesis we have $z\leqslant y^2$. Calculation gives

$$ \begin{equation*} s_W(E)=\frac{d+3e}{4d-1}\leqslant\frac{6d^2-10d+1}{8d-2}, \qquad s(E,F)=d-y-1. \end{equation*} \notag $$
Since $s(E,F)\leqslant s_W(E)$, we have $y\geqslant(2d^2+1)/(8d-2)>d/2$. Maximizing $\operatorname{ch}_3(E/F)$ we obtain
$$ \begin{equation*} e\leqslant (d-y)^2-2(d-y)+y^2-2y+C, \end{equation*} \notag $$
where $C$ is a constant depending on whether $d$ and $y$ are integers or not. For $y\in\mathbb{Z}$ or $y\in\frac 12+\mathbb{Z}$ the right-hand side of the inequality is a quadratic function of $y$ with the minimum at the point $y=\frac d2$. Therefore, the maximum is attained at $y=-\frac 12$ or $y=-1$. For $y=-\frac 12$ using calculation we obtain
$$ \begin{equation*} e\leqslant d^2-d+\frac{11}{12}<d^2-2d+\frac 16, \end{equation*} \notag $$
and for $y=-1$ we find that
$$ \begin{equation*} e\leqslant d^2+\frac{29}{12}<d^2-2d+\frac 16. \end{equation*} \notag $$

(2) Suppose that $c=0$, $d\leqslant-\frac 52$ and $e\geqslant d^2$. As in the proof of Lemma 3.11, we have $W_{0,-1}(E)<0$, which implies that $\operatorname{ch}_1(F)=-H$. Suppose $\operatorname{ch}(F)=(2,-H,yH^2,z)$. By the induction hypothesis we have $z\leqslant y^2-2y+\frac{5}{12}$. Calculation gives

$$ \begin{equation*} s_W(E)=\frac{3e}{4d}\leqslant\frac 34d\quad\text{and} \quad s(E,F)=d-y, \end{equation*} \notag $$
hence $y\geqslant d/4>d/2+1/2$. Maximizing $\operatorname{ch}_3(E/F)$, we find that
$$ \begin{equation*} e\leqslant 2y^2-(2d+2)y+C, \end{equation*} \notag $$
where $C$ is a constant depending on whether or not $d$ and $y$ are integers. For $y\in\mathbb{Z}$ or $y\in\frac 12+\mathbb{Z}$ the right-hand side of the inequality is a quadratic function of $y$ with the minimum at $y=d/2+1/2$. Therefore, the maximum is attained at $y=0$ or $y=-\frac 12$. For $y=-\frac 12$ we obtain $e\leqslant d^2+d+2<d^2$, which implies that $y=0$. It remains to note that the equality $\operatorname{Ext}^1(F,E/F)=0$ holds, which is verified by the exact triples defining $F$ and $E/F$.

(3) and (4). These are direct consequences of statements (1) and (2), respectively.

Theorem 3.1 is proved.

§ 4. Infinite series of components of moduli spaces of semistable sheaves of rank $2$ on the varieties $X_1$, $X_2$, $X_4$ and $X_5$

4.1. First series

In this subsection we consider extensions of one of the following two forms.

(1) Extensions to $X=X_1$, $X_2$, $X_4$ or $X_5$ of the form

$$ \begin{equation} \begin{gathered} \, 0\to\mathcal O_X(-n)^{\oplus2}\to E\to\mathcal O_S(m)\to 0, \qquad S\in|\mathcal O_X(k)|, \\ \text{where} \qquad k\geqslant1, \qquad n=\biggl\lceil\frac k2\biggr\rceil\quad\text{and} \quad m+n<0, \end{gathered} \end{equation} \tag{4.1} $$
(2) or an extension to the quadric $X=X_2$ of the form
$$ \begin{equation} \begin{gathered} \, 0\to\mathcal O_X(-1)^{\oplus 2}\to E\to\mathcal I(m)\to 0, \qquad S\in|\mathcal O_X(1)|, \\ \text{where } \mathcal I=\mathcal I_{\mathbb{P}^1,S}, \qquad m<0, \end{gathered} \end{equation} \tag{4.2} $$
$\mathbb{P}^1$ is a projective line on a two-dimensional quadric $S$ and the sheaf $\mathcal{I}$ is described in (2.10). This section deals mainly with the extensions (4.1). In particular, for $E$ from (4.1) we have
$$ \begin{equation} \begin{gathered} \, \operatorname{rk}(E)=2, \qquad c_1(E)=0, \qquad c_2(E)=\biggl(\frac{k^2}{4}-km\biggr)H^2, \\ c_3(E)=\biggl(\frac{k^3}{4}-k^2m+km^2\biggr)H^3 \quad \text{if } k\ \text{is even} \end{gathered} \end{equation} \tag{4.3} $$
and
$$ \begin{equation} \begin{gathered} \, c_1(E)=-H, \qquad c_2(E)=\biggl(\frac{(k-1)^2}{4}-km\biggr)H^2, \\ c_3(E)=\biggl(\frac{k(k-1)^2}{4} +km-k^2m+km^2\biggr)H^3 \quad \text{if } k\ \text{is odd}, \end{gathered} \end{equation} \tag{4.4} $$
where $H$ is the cohomology class of a hyperplane section $X$.

It is well known and verified by standard calculation that

$$ \begin{equation} h^i(\mathcal O_X(\ell))=0, \qquad \ell\in\mathbb{Z}, \qquad i=1,2. \end{equation} \tag{4.5} $$
Now assume that
$$ \begin{equation} \text{the sheaf }E\text{ in } (4.1)\text{ is stable}. \end{equation} \tag{4.6} $$
In this case, from (2.10), (4.1), (4.2) and (4.5), the exact triples
$$ \begin{equation} 0\to\mathcal O_X(-k)\to\mathcal O_X\to\mathcal O_S\to0 \end{equation} \tag{4.7} $$
and
$$ \begin{equation} 0\to\mathcal I\to\mathcal O_S\to\mathcal O_{\mathbb{P}^1}\to0, \qquad X=X_2, \end{equation} \tag{4.8} $$
the Serre–Grothendieck duality and the stability of $E$ we obtain the following equalities:

(a) for the extension (4.1),

$$ \begin{equation} \begin{gathered} \, \hom(E,E)=\hom(\mathcal O_S(m),\mathcal O_S(m))=\hom(E,\mathcal O_S(m))=1, \\ \operatorname{ext}^1(\mathcal O_S(m),\mathcal O_S(m))=N \end{gathered} \end{equation} \tag{4.9} $$
and
$$ \begin{equation} \begin{gathered} \, \hom(\mathcal O_X(-n)^{\oplus 2},\mathcal O_S(m))=0, \\ \begin{aligned} \, & \operatorname{ext}^1(\mathcal O_X(-n)^{\oplus 2},E)=\operatorname{ext}^1(\mathcal O_X(-n) ^{\oplus 2},E)=\operatorname{ext}^1(\mathcal O_X(-n)^{\oplus 2},\mathcal O_S(m)) \\ &\qquad=\operatorname{ext}^1(\mathcal O_X(-n)^{\oplus 2},\mathcal O_X(-n)^{\oplus 2})= \operatorname{ext}^i(\mathcal O_S(m),\mathcal O_X(-n)^{\oplus 2})=0, \qquad i=0,2; \end{aligned} \end{gathered} \end{equation} \tag{4.10} $$

(b) for the extension (4.2),

$$ \begin{equation} \begin{gathered} \, \hom(E,E)=\hom(\mathcal O_S(m),\mathcal O_S(m))=\hom(E,\mathcal O_S(m))=1, \\ \operatorname{ext}^1(\mathcal O_S(m),\mathcal O_S(m))=4, \end{gathered} \end{equation} \tag{4.11} $$
$$ \begin{equation} \begin{gathered} \, \hom(\mathcal I(m),\mathcal I(m))=\hom(E,\mathcal I(m))=1, \qquad \hom(\mathcal O_X(-n)^{\oplus2},\mathcal I(m))=0, \\ \operatorname{ext}^i(\mathcal O_X(-1)^{\oplus2},\mathcal I(m))=0, \qquad i=0,1, \end{gathered} \end{equation} \tag{4.12} $$
$$ \begin{equation} \hom(\mathcal O_X(-1)^{\oplus2},\mathcal I(1))=4, \qquad \hom(\mathcal S(-1),\mathcal I(1))=7, \qquad \operatorname{ext}^1(\mathcal O_X^{\oplus2},\mathcal I(1))=0, \end{equation} \tag{4.13} $$
and
$$ \begin{equation} \begin{gathered} \, \hom(\mathcal O_S,\mathcal O_{\mathbb{P}^1})=1, \qquad \operatorname{ext}^1(\mathcal O_S,\mathcal O_{\mathbb{P}^1})=k+1, \qquad \operatorname{ext}^1(\mathcal O_S,\mathcal O_S)=N, \\ \operatorname{ext}^i(\mathcal I(m),\mathcal O_X(-n)^{\oplus 2})=0, \qquad i=0,2. \end{gathered} \end{equation} \tag{4.14} $$
Let $\mathbb{P}^N=|\mathcal{O}_X(k)|$, let $\mathbb{S}\subset\mathbb{P}^N\times X$ be the universal family of surfaces of degree $k$ in $X$, and let $\mathbb{P}^N\times X\xrightarrow{p} \mathbb{P}^N$ be the projection. Correspondingly, using the notation of the triple (2.9), let $\mathbb{G}\times X_2\xrightarrow{\widetilde{p}}\mathbb{G}$ be the projection. By the smoothness of $\mathbb{P}^N$ and $\mathbb{G}$ it follows from (4.10)(4.14), [20], Theorem 1.4, and [5], Satz 3, (ii), that locally free sheaves $\mathcal{A}$ and $\widetilde{\mathcal{A}}$ are well defined on $\mathbb{P}^N$ and $\mathbb{G}$, respectively:
$$ \begin{equation} \begin{gathered} \, \mathcal A=\mathcal E xt_p^1(\mathcal O_{\mathbb{P}^N} \boxtimes\mathcal O_X(m)|_{\mathbb{S}},\mathcal O_{\mathbb{P}^N} \boxtimes\mathcal O_X(-n)), \\ \widetilde{\mathcal A}=\mathcal E xt_{\widetilde{p}}^1(\mathbb{I}\otimes\mathcal O_{ \mathbb{G}}\boxtimes\mathcal O_{X_2}(m),\mathcal O_{\mathbb{G}}\boxtimes\mathcal O_{X_2}(-1)). \end{gathered} \end{equation} \tag{4.15} $$
The construction of these sheaves commutes with the base change:
$$ \begin{equation} \begin{gathered} \, \tau_S\colon \mathcal A\otimes\mathbf k_{S}\xrightarrow{\cong}\operatorname{Ext}^1(\mathcal O_S(m),\mathcal O_X(-n)), \qquad S\in|\mathcal O_X(k)|, \\ \widetilde{\tau}_x\colon \widetilde{\mathcal A}\otimes\mathbf k_x\xrightarrow{\cong} \operatorname{Ext}^1(\mathcal I(m),\mathcal O_X(-1)), \qquad x\in\mathbb{G}, \quad S=S_x, \end{gathered} \end{equation} \tag{4.16} $$
and we have
$$ \begin{equation} \begin{gathered} \, \mathcal E xt_p^0(\mathcal O_{\mathbb{P}^N}\boxtimes\mathcal O_X(m)|_{\mathbb{S}},\mathcal O_{\mathbb{P}^N} \boxtimes\mathcal O_X(-n))=0, \\ \mathcal E xt_{\widetilde{p}}^0(\mathbb{I}\otimes\mathcal O_{\mathbb{G}}\boxtimes \mathcal O_{X_2}(m),\mathcal O_{\mathbb{G}}\boxtimes\mathcal O_{X_2}(-1))=0. \end{gathered} \end{equation} \tag{4.17} $$
Consider the varieties $P=\mathbb{P}(\mathcal{A}^{\vee})$, $X_P=P\times X$, $\widetilde{P}=\mathbb{P}(\widetilde{\mathcal{A}}^{\vee})$ and $X_{\widetilde{P}}= \widetilde{P}\times X_2$, with the projections $P\xrightarrow{\rho}\mathbb{P}^N$, $\widetilde{P}\xrightarrow{\widetilde{\rho}}\mathbb{G}$, $P\xleftarrow{p_P}X_P\xrightarrow{\rho_P}\mathbb{P}^N\times X$ and $\widetilde{P}\xleftarrow{\widetilde{p}_P}X_{\widetilde{P}} \xrightarrow{\widetilde{\rho}_P}\mathbb{G}\times X_2$. From (4.16), (4.17) and [20], Corollary 4.5, it follows that there are universal families of extensions on $X_P$ and $X_{\widetilde{P}}$, respectively,
$$ \begin{equation*} 0\to p_P^*\mathcal O_P(1)\otimes(\mathcal O_{\mathbb{P}^N}\boxtimes\mathcal O_X(-n))\to\mathcal V\to \rho_P^*(\mathcal O_{\mathbb{P}^N}\boxtimes\mathcal O_X(m)|_{\mathbb{S}})\to0 \end{equation*} \notag $$
and
$$ \begin{equation*} 0\to\widetilde{p}_P^{\,*}\mathcal O_{\widetilde{P}}(1)\otimes(\mathcal O_{\mathbb{G}} \boxtimes\mathcal O_{X_2}(-1))\to\widetilde{\mathcal V}\to\widetilde{\rho}_P^{\,*}(\mathbb{I}\otimes \mathcal O_{\mathbb{G}}\boxtimes\mathcal O_{X_2}(m))\to0. \end{equation*} \notag $$
Applying the functors $\rho_{P*}$ and $\widetilde{\rho}_{P*}$ to these triples, respectively, we obtain the universal extensions on $\mathbb{P}^N\times X$ and on $\mathbb{G}\times X$, where $\mathcal{W}=\rho_{P*}\mathcal{V}$ and $\widetilde{\mathcal{W}}=\widetilde {\rho}_{P*}\widetilde{\mathcal{V}}$:
$$ \begin{equation} \begin{gathered} \, 0\to p^*\mathcal A^{\vee}\otimes(\mathcal O_{\mathbb{P}^N}\boxtimes\mathcal O_X(-n))\to\mathcal W\to\mathcal O_{\mathbb{P}^N}\boxtimes \mathcal O_X(m)|_{\mathbb{S}}\to0, \\ 0\to\widetilde{p}^*\widetilde{\mathcal A}^{\vee}\otimes(\mathcal O_{\mathbb{G}}\boxtimes\mathcal O_{X_2}(-1)) \to\widetilde{\mathcal W}\to\mathbb{I}\otimes\mathcal O_{\mathbb{G}}\boxtimes\mathcal O_{X_2}(m)\to0. \end{gathered} \end{equation} \tag{4.18} $$

Note that the extensions (4.1) and (4.2) are specified by elements $\xi\mkern-1mu\!\in\!\operatorname{Ext}^1\mkern-1mu(\mathcal{O}_S(m),\mathcal{O}_X(-n)^{\oplus2})$ and $\widetilde{\xi}\in\operatorname{Ext}^1(\mathcal{I}(m), \mathcal{O}_{X_2}(-1)^{\oplus2})$, respectively, considered as homomorphisms

$$ \begin{equation} \xi\colon \operatorname{Ext}^1(\mathcal O_S(m),\mathcal O_X(-n))^{\vee}\to\mathbf k^2 \quad\text{and}\quad \widetilde{\xi}\colon \operatorname{Ext}^1(\mathcal I(m),\mathcal O_{X_2}(-1))^{\vee}\to\mathbf k^2. \end{equation} \tag{4.19} $$
We carry out the further argument in this subsection for the upper exact triple (4.18). (For the lower triple (4.18) the argument is quite similar: see § 4.2.) By universality this triple, as restricted to $\{S\}\times X$, in view of the base change (4.16) is included, together with the extension (4.1), where $E=E_{\xi}$, in the commutative pushout diagram
$(4.20)$
Note that the homomorphism $\xi$ in (4.19) is nonzero since the extension (4.1) is nontrivial. Moreover, if $E_{\xi}$ is stable, then $\xi$ is an epimorphism. Indeed, if $\xi$ has rank 1, then the diagram (4.20) goes through a subextension $0\to\mathcal{O}_X(-n)\to E'\to\mathcal{O}_S(m)\to0$ of the extension (4.1), which implies the existence of a homomorphism $E_{\xi}\twoheadrightarrow \mathcal{O}_X(-n)$, contrary to the stability of $E_{\xi}$.

Suppose that $\xi$ in (4.20) is an epimorphism. The class $[\xi]$ of the epimorphism $\xi$ modulo the automorphisms of the space $\mathbf{k}^2$ is a point of the Grassmannian $\mathrm{Gr}(\operatorname{Ext}^1(\mathcal{O}_S(m),\mathcal{O}_X(-n))^{\vee},2)$. Therefore, let us consider the Grassmannization

$$ \begin{equation} \pi\colon {\mathcal G}r(\mathcal A^{\vee},2)\to\mathbb{P}^N \end{equation} \tag{4.21} $$
of two-dimensional quotient spaces of the bundle $\mathcal{A}^{\vee}$ on $\mathbb{P}^N$ and its open subsets
$$ \begin{equation} \mathcal Y_{k,m,n}:=\{[\xi]\in{\mathcal G}r(\mathcal A^{\vee},2)\mid E_{\xi}\ \text{is stable}\} \end{equation} \tag{4.22} $$
and
$$ \begin{equation} \mathcal R_{k,m,n}:=\{[\xi]\in{\mathcal G}r(\mathcal A^{\vee},2)\mid E_{\xi}\ \text{is reflexive}\}. \end{equation} \tag{4.23} $$

Theorem 4.1. There are embeddings of dense open sets

$$ \begin{equation} \mathcal R_{k,m,n}\subset\mathcal Y_{k,m,n}\subset{\mathcal G}r(\mathcal A^{\vee},2). \end{equation} \tag{4.24} $$

Proof. (1) First, let us check the reflexivity of a general sheaf $E$ in the triple (4.1). (In the remaining cases the proofs are completely similar.) Namely, applying the functor $\mathbf{R}\mathcal{H}om(-,\mathcal{O}_X)$ to the triple (4.1) and taking into account that $\mathcal{E}xt^1(\mathcal{O}_S(m),\mathcal{O}_X)\cong\mathcal{O}_S(k-m)$, we obtain the exact sequence
$$ \begin{equation} 0\to E^{\vee}\to\mathcal O_X(n)^{\oplus2}\to\mathcal O_S(m)\to\tau\to0, \qquad \tau:=\mathcal E xt^1(E,\mathcal O_X). \end{equation} \tag{4.25} $$
The long exact sequence defined by the spectral sequence of local and global Exts
$$ \begin{equation*} E^{pq}_2=H^p(\mathcal E xt_{\mathcal O_X}^q(\mathcal O_S(m),\mathcal O_X(-n)^{\oplus2 })) \Longrightarrow\operatorname{Ext}^{p+q}(\mathcal O_S(m),\mathcal O_X(-n)^{\oplus2}), \end{equation*} \notag $$
gives an isomorphism
$$ \begin{equation} \begin{aligned} \, \notag \operatorname{Ext}^1(\mathcal O_S(m),\mathcal O_X(-n)^{\oplus2}) &\xrightarrow{\cong}H^0(\mathcal E xt^1(\mathcal O_S(m),\mathcal O_X(-n)^{\oplus2})) \\ &\cong H^0(\mathcal O_S(k-m-n)^{\oplus2}). \end{aligned} \end{equation} \tag{4.26} $$
Under this isomorphism the element $\xi\in\operatorname{Ext}^1(\mathcal{O}_S(m),\mathcal{O}_X(-n) ^{\oplus2}))$, defining the extension (4.1), corresponds to a section $s_{\xi}\in H^0(\mathcal{O}_S(k-m-n)^{\oplus2})$, the scheme of zeros $Z_{\xi}=(s_{\xi})_0$ of which, due to the ampleness of the sheaf $\mathcal{O}_S(k-m-n)$ for $k\geqslant1$ and $m+n<0$, is zero-dimensional for $\xi$ belonging to an open dense subset of $\operatorname{Ext}^1(\mathcal{O}_S(m),\mathcal{O}_X(-n)^{\oplus2}))$. A standard local calculation shows that $\operatorname{Sing} E=Z_{\xi}$, so $\tau$ is an Artinian sheaf and thus $\mathcal{E} xt^i(\tau,\mathcal{O}_X)=0$ for $i\leqslant2$. Therefore, applying the functor $\mathbf{R}\mathcal{H}om(-,\mathcal{O}_X)$ to (4.25) and taking the isomorphism $\mathcal{E}xt^1(\mathcal{O}_S({k-m}),\mathcal{O}_X)\cong\mathcal{O}_S(m)$ into account, we obtain an exact triple $0\to\mathcal{O}_X(-n)^{\oplus 2}\to E^{\vee\vee}\to\mathcal{O}_S(m)\to 0$ and its isomorphism with the triple (4.1) via the functor $(-)^{\vee\vee}$. This implies the required isomorphism $E\cong E^{\vee\vee}$.

(2) Let us prove the stability of an arbitrary sheaf $E\in\mathcal{R}_{k,m,n}$. Recall that in (4.1) we have $k\geqslant1$, $n=\lceil\frac k2\rceil\geqslant1$ and $m<-n\leqslant1$. This implies that

$$ \begin{equation} -1\leqslant c_1(E)\leqslant0 \end{equation} \tag{4.27} $$
and
$$ \begin{equation} h^0(E)=0. \end{equation} \tag{4.28} $$
Assume that $E$ is unstable. In view of (4.27) the maximal destabilizing rank 1 subsheaf $F$ of $E$ has the form $F=\mathcal{I}_{Z,\mathbb{P}^3}(q)$, $q\geqslant0$. By virtue of the reflexivity of $E$ the monomorphism $F\to E$ extends to a monomorphism $\mathcal{O}_{\mathbb{P}^3}(q) \cong F^{\vee\vee}\overset{s}{\to}E$, that is, a section $0\ne s\in h^0(E(-q)) $. This contradicts (4.28) since $q\geqslant0$.

The theorem is proved.

Consider the modular morphism of the variety $\mathcal{Y}_{k,m,n}$ into the Gieseker–Maruyama moduli scheme $M_X(v)$:

$$ \begin{equation} \Theta\colon \mathcal Y_{k,m,n}\to M_X(v), \qquad [\xi]\mapsto[E_{\xi}], \end{equation} \tag{4.29} $$
where $[E_{\xi}]$ is the isomorphism class of the stable sheaf $E_{\xi}$ obtained from the point $\xi$ as the lower extension in the diagram (4.20), and $v=\mathrm{ch}(E)$ is determined by formulae (2.4) and (4.3)(4.4).

Note that by virtue of the first equality in (4.10) the sheaf $[E=E_{[\xi]}]\in \Theta(\mathcal{Y}_{k,m,n})$ has a unique subsheaf $\mathcal{O}_X(-n)^{\oplus2}$, and so the extension (4.1) is uniquely defined, that is, the class $[\xi]$ can uniquely be recovered from the point $[E]$. Therefore,

$$ \begin{equation} \Theta\colon \mathcal Y_{k,m,n}\to M_{k,m,n}:=\Theta(\mathcal Y_{k,m,n}) \text{ is a bijection} \end{equation} \tag{4.30} $$
and
$$ \begin{equation} \begin{split} & \dim M_{k,m,n}=\dim\mathcal Y_{k,m,n}=\dim\operatorname{Gr}(\operatorname{Ext}^1(\mathcal O_S(m),\mathcal O_X(-n))^{\vee},2) \\ &\qquad\qquad +\dim|\mathcal O_X(k)|=\operatorname{ext}^1(\mathcal O_S(m),\mathcal O_X(-n)^{\oplus2})-4+N. \end{split} \end{equation} \tag{4.31} $$
Applying the bifunctor $\mathbf{R}\operatorname{Hom}(-,-)$ to the triple (4.1) and taking (4.9) and (4.10) into account, for $[E]\in\dim\mathcal{Y}_{k,m,n}$ we obtain a commutative diagram
$(4.32)$
From this diagram and the equality $\mathrm{Ext}^1(\mathcal{O}_S(m),\mathcal{O}_S(m))=N$ (see (4.9)) we obtain
$$ \begin{equation} \operatorname{Ext}^1(E,E)=\operatorname{Ext}^1(\mathcal O_S(m),\mathcal O_X(-n)^{\oplus2})-4+N, \end{equation} \tag{4.33} $$
hence from (4.31) we find that $\dim\Theta(\mathcal{Y}_{k,m,n})=\mathrm{ext}^1(E,E)= \dim_{[E]} M_X(v)$. Taking Theorem 4.1 into account, this means that $M_{k,m,n}$ is a smooth open subset of an irreducible component $\overline{M}_{k,m,n}$ of the moduli scheme $M_X(v)$. From this and (4.30), according to Theorem 2.16 in [28], Ch. 2, § 4.4, it follows that
$$ \begin{equation} \Theta\colon \mathcal Y_{k,m,n}\to M_{k,m,n} \text{ is an isomorphism}, \end{equation} \tag{4.34} $$
that is, $M_{k,m,n}$ is a smooth rational variety which is isomorphic in view of Theorem 4.1 to a dense open subset of the smooth variety ${\mathcal{G}}r(\mathcal{A} ^{\vee},2)$. At the same time, it follows directly from the diagram (4.20) that there is a universal family of sheaves on $M_{k,m,n}$. Formulae for the dimensions of the varieties $M_{k,m,n}$ as functions of $k$, $m$ and $n$ are obtained by taking into account the Riemann–Roch theorem for $X_5$, formulae (4.5), (4.7) and (4.31), and the standard resolutions for the sheaves $\mathcal{O}_{X_2}$ and $\mathcal{O}_{X_4}$:
$$ \begin{equation} \begin{gathered} \, 0\to\mathcal O_{\mathbb{P}^4}(-2)\to\mathcal O_{\mathbb{P}^4}\to\mathcal O_{X_2}\to0, \\ 0\to\mathcal O_{\mathbb{P}^5}(-4)\to\mathcal O_{\mathbb{P}^5}(-2)^{\oplus2}\to\mathcal O_{\mathbb{P}^5}\to\mathcal O_{X_4}\to0. \end{gathered} \end{equation} \tag{4.35} $$
Moreover, in view of Theorem 4.1 the reflexive sheaves in $M_{k,m,n}$ form a dense open set
$$ \begin{equation} R_{k,m,n}:=\Theta(\mathcal R_{k,m,n})\cong\mathcal R_{k,m,n}. \end{equation} \tag{4.36} $$

Thus, the following theorem holds.

Theorem 4.2. Let $X$ be one of the Fano varieties $X_1$, $X_2$, $X_4$ and $X_5$. For an arbitrary natural number $k$ and integers $m$ and $n=\lceil\frac k2\rceil$ such that $m+n<0$ let $M_X(v)$ be the Gieseker–Maruyama moduli scheme of semistable sheaves $E$ on $X$ with Chern character $v=\mathrm{ch}(E)$, determined by (2.4), (4.3) and (4.4). Then the following statements are true.

(i) The family

$$ \begin{equation*} M_{k,m,n}=\{[E]\in M_X(v)\mid E\textit{ is a stable extension (4.1), where}\ S\in|\mathcal O_X(k)|\} \end{equation*} \notag $$
is a smooth dense open subset of an irreducible rational component of the scheme $M_X(v)$, and there is an isomorphism (4.34) in which $\mathcal{Y}_{k,m,n}$ is the open subset, defined in (4.22), of the Grassmannization $\pi\colon \mathcal{G}r(\mathcal{A}^{\vee},2)\to\mathbb{P}^N$, where $\mathcal{A}$ is the locally free sheaf on $\mathbb{P}^N=|\mathcal{O}_X(k)|$ given by the first formula in (4.15); in this case $M_{k,m,n}$ is a fine moduli space, and the reflexive sheaves form the dense open subset $R_{k,m,n}$ defined in (4.36).

(ii) The dimension of the variety $M_{k,m,n}$ is given by the following formulae:

(iii) The dimension of the variety $M_{k,m,n}$ coincides with the virtual dimension of the scheme $M_X(v)$, that is, $\operatorname{ext}^2(E,E)=0$ for $[E]\in M_{k,m,n}$, if and only if $k<i$ and $m>k-n-i$, where $i$ is the index of the Fano variety $X$.

Proof. Statements (i) and (ii) of the theorem were proved above. Statement (iii) is proved by a direct calculation by analogy with formulae (4.10). The theorem is proved.

Remark 4.1. Direct checking using formulae (4.3) and (4.4) for the Chern classes of the sheaf $[E]\in M_{k,m,n}$ shows that

$$ \begin{equation*} \frac{c_3(E)}{H^3}< \frac 4k\biggl(\frac{c_2(E)}{H^2}\biggr)^2 \end{equation*} \notag $$
in the case of even $k$, and, respectively,
$$ \begin{equation*} \frac{c_3(E)}{H^3}<\frac{4k}{(k-1)^2}\biggl(\frac{c_2(E)}{H^2}\biggr)^2 \end{equation*} \notag $$
in the case of odd $k \geqslant 3$. For $k=1$
$$ \begin{equation*} \frac{c_3(E)}{H^3} = \biggl(\frac{c_2(E)}{H^2} \biggr)^2. \end{equation*} \notag $$

Remark 4.2. On the quadric $X=X_2$ we consider a nontrivial extension of the form

$$ \begin{equation} \begin{gathered} \, 0\to\mathcal S(-n)\to E\to\mathcal O_S(m)\to 0, \qquad S\in|\mathcal O_X(k)|, \\ k\geqslant1, \qquad n=\biggl\lfloor\frac k2\biggr\rfloor+1, \qquad m\leqslant-n, \end{gathered} \end{equation} \tag{4.37} $$
where $\mathcal{S}$ is the spinor bundle on $X$. Let $M_X(v)$ be the Gieseker–Maruyama moduli scheme of semistable sheaves $E$ with Chern character $v=\mathrm{ch}(E)$ defined by (2.4), (2.7) and (4.37). From (4.37) it follows that $E$ satisfies (4.9), (4.10), (4.33) and the diagram (4.32) throughout which $\mathcal{O}_X(-n)^{\oplus2}$ is replaced by $\mathcal{S}(-n)$, and the left vertical triple in (4.32) is replaced by the triple $\mathbf{k}\rightarrowtail\mathbf{k}\twoheadrightarrow0$. Correspondingly, in the notation of the first formula in (4.15), the following locally free sheaf is defined on $\mathbb{P}^N$:
$$ \begin{equation} \mathcal A_{\mathcal S}=\mathcal E xt_p^1(\mathcal O_{\mathbb{P}^N}\boxtimes\mathcal O_X(m)|_{\mathbb{S}},\mathcal O_{\mathbb{P}^N}\boxtimes \mathcal S(-n)). \end{equation} \tag{4.38} $$
In addition, replacing the sheaf $\mathcal{O}_X(-n)^{\oplus2}$ in (4.21)(4.23) by $\mathcal{S}(-n)$ we obtain the projectivization $\pi\colon \mathbb{P}(\mathcal{A}_{\mathcal{S}}^{\vee})\to\mathbb{P}^N$ and its dense open subsets $\mathcal{Y}_{\mathcal{S},k,m,n}$ and $\mathcal{R}_{\mathcal{S},k,m,n}$ of stable and reflexive sheaves respectively. For them, by analogy with (4.24), we have the embeddings of dense open sets
$$ \begin{equation} \mathcal R_{\mathcal S,k,m,n}\subset\mathcal Y_{\mathcal S,k,m,n}\subset{\mathcal G}r(\mathcal A_{\mathcal S}^{\vee},2) \end{equation} \tag{4.39} $$
and, as in (4.34), an isomorphism
$$ \begin{equation} \Theta\colon \mathcal Y_{\mathcal S,k,m,n}\xrightarrow{\cong} M_{k,m,n}:=\Theta(\mathcal Y_{\mathcal S,k,m,n}), \qquad [\xi]\mapsto[E_{\xi}]. \end{equation} \tag{4.40} $$
Repeating the corresponding steps of the proof of Theorem 4.2 for the varieties indicated, we obtain the following theorem.

Theorem 4.3. Let $X=X_2$ be the three-dimensional quadric. Then in the notation of Remark 4.2 the following statements are true.

(i) The family

$$ \begin{equation*} M_{k,m,n}=\{[E]\in M_X(v)\mid E\textit{ is a stable extension (4.37), where}\ S\in|\mathcal O_X(k)|\} \end{equation*} \notag $$
is a smooth dense open subset of an irreducible rational component of scheme $M_X(v)$, and there is an isomorphism $\Theta$ in (4.40), where $\mathcal{Y}_{\mathcal{S},k,m,n}$ is an open subset (defined in Remark 4.2) of the projectivization $\pi\colon \mathbb{P}(\mathcal{A}_{\mathcal{S}}^{\vee})\to\mathbb{P}^N$, where $\mathcal{A}_{\mathcal{S}}$ is the locally free sheaf on $\mathbb{P}^N=|\mathcal{O}_X(k)|$ given by (4.38); in addition, $M_{k,m,n}$ is a fine moduli space, and reflexive sheaves form a dense open set $R_{\mathcal{S},k,m,n}=\Theta(\mathcal{R}_{\mathcal{S},k,m,n})$ in $M_{k,m,n}$, where $\mathcal{R}_{\mathcal{S},k,m,n}$ was defined in Remark 4.2.

(ii) The dimension of the variety $M_{k,m,n}$ is given by the formula

$$ \begin{equation*} \begin{aligned} \, \dim M_{k,m,n} &=4\binom{k-m-n+3}{3}-4\binom{k-m-n+3}{3} \\ &\qquad-4\binom{-m-n+3}{3}+4\binom{-m-n +3}{3}+3. \end{aligned} \end{equation*} \notag $$

4.2. The second series

In this subsection we treat in more detail the extensions (4.2) on the quadric $X=X_2$.

First we find the dimension of the space $\operatorname{Ext}^1(\mathcal{I}(m),\mathcal{I}(m))$, which we need below. To do this we apply the functor $\mathbf{R}\operatorname{Hom}(-,\mathcal{I}(1))$ to the triple (2.10):

$$ \begin{equation*} \begin{aligned} \, 0 &\to\operatorname{Hom}(\mathcal I(1),\mathcal I(1))\to\operatorname{Hom}(\mathcal O_X^{\oplus2},\mathcal I(1))\to\operatorname{Hom}(\mathcal S(-1),\mathcal I(1)) \\ & \to\operatorname{Ext}^1(\mathcal I(1),\mathcal I(1))\to\operatorname{Ext}^1(\mathcal O_X^{\oplus2},\mathcal I(1)). \end{aligned} \end{equation*} \notag $$
From this and (4.12) and (4.13) we find that
$$ \begin{equation} \operatorname{ext}^1(\mathcal I(m),\mathcal I(m))=\operatorname{ext}^1(\mathcal I(1),\mathcal I(1))=4. \end{equation} \tag{4.41} $$

Consider now the lower triple (4.18). By universality and in view of the base change (4.16) the restriction of this triple to $\{x\}\times X$, $x\in\mathbb{G}$ can be included, together with the extension (4.2), where $\widetilde{\xi}\colon \operatorname{Ext}^1(\mathcal{I}(m),\mathcal{O}_X(-1))^{\vee} \to \mathbf{k}^2$ is the second homomorphism in (4.19) and $E=E_{\widetilde{\xi}}$, in a commutative diagram similar to (4.20):

$(4.42)$
Moreover, as in (4.20), if $E_{\widetilde{\xi}}$ is stable, then $\widetilde{\xi}$ is an epimorphism. Therefore, by analogy with (4.22) and (4.23), consider the Grassmannization
$$ \begin{equation*} \widetilde{\pi}\colon {\mathcal G}r(\widetilde{\mathcal A}^{\vee},2)\to\mathbb{G} \end{equation*} \notag $$
of the two-dimensional quotient spaces of the bundle $\widetilde{\mathcal{A}}^{\vee}$ on $\mathbb{G}$, defined in (4.15), and its open subsets
$$ \begin{equation} \widetilde{\mathcal Y}_m:=\{[\widetilde{\xi}]\in{\mathcal G}r(\widetilde{\mathcal A}^{\vee},2)\mid E_{\widetilde{\xi}} \text{ is stable}\} \end{equation} \tag{4.43} $$
and
$$ \begin{equation} \widetilde{\mathcal R}_m:=\{[\widetilde{\xi}]\in{\mathcal G}r(\widetilde{\mathcal A}^{\vee},2)\mid E_{\widetilde{\xi}}\ \text{is reflexive}\}. \end{equation} \tag{4.44} $$
Repeating the proof of Theorem 4.1 for $E$ in the triple (4.2) we obtain the embeddings of dense open subsets
$$ \begin{equation} \widetilde{\mathcal R}_m\subset\widetilde{\mathcal Y}_m\subset{\mathcal G}r(\widetilde{\mathcal A}^{\vee},2). \end{equation} \tag{4.45} $$
As in (4.29), we have the modular morphism $\Theta\colon \widetilde{\mathcal{Y}}_m\to M_X(v)$, $[\xi]\mapsto[E_{\widetilde{\xi}}]$, where $[E_{\widetilde{\xi}}]$ is the isomorphism class of the bundle $E_{\widetilde{\xi}}$ obtained from the point $\widetilde{\xi}$ as the lower extension in the diagram (4.42). Assuming that $\widetilde{M}_m=\Theta(\widetilde{\mathcal{Y}}_m)$, as in (4.30) and (4.31), we have a bijection
$$ \begin{equation} \Theta\colon \widetilde{\mathcal Y}_m\xrightarrow{\simeq}\widetilde{M}_m \end{equation} \tag{4.46} $$
and the equality
$$ \begin{equation} \begin{aligned} \, \notag \dim\widetilde{M}_m &=\dim\widetilde{\mathcal Y}_m=\dim\operatorname{Gr}(\operatorname{Ext}^1(\mathcal I(m), \mathcal O_X(-1))^{\vee},2)+\dim\mathbb{G} \\ &=2\operatorname{ext}^1(\mathcal I(m),\mathcal O_X(-1))-4+4 =\operatorname{ext}^1(\mathcal I(m),\mathcal O_X(-1)^{\oplus2}). \end{aligned} \end{equation} \tag{4.47} $$
Applying the bifunctor $\mathbf{R}\operatorname{Hom}(-,-)$ to the triple (4.2) and taking (4.9)(4.12) into account we obtain the commutative diagram
$(4.48)$
From this diagram and equalities (4.41) and (4.47) we obtain
$$ \begin{equation} \operatorname{ext}^1(E,E)=\operatorname{ext}^1(\mathcal I(m),\mathcal O_X(-n)^{\oplus2})=\dim\widetilde{M}_m. \end{equation} \tag{4.49} $$
From here, according to deformation theory, in view of the stability of $E$ it follows that $\widetilde{M}_{k,m,n}$ is a smooth open subset of an irreducible component of the moduli scheme $M_X(v)$. Therefore, from (4.46) it follows as in (4.34) that
$$ \begin{equation} \Theta\colon \widetilde{\mathcal Y}_m\to\widetilde{M}_m \text{ is an isomorphism}, \end{equation} \tag{4.50} $$
that is, $\widetilde{M}_m$ is a smooth rational variety, which is isomorphic by (4.45) to a dense open subset of the smooth variety ${\mathcal{G}}r(\widetilde{\mathcal{A}}^{\vee},2)$. Moreover, $\widetilde{M}_m$ is also a fine moduli space. Formulae for the dimensions of the varieties $\widetilde{M}_m$ as functions of $m$ are obtained by taking (4.5)(4.8) and (4.47) into account, by analogy with the formulae for the dimensions of the $M_m$. Moreover, by (4.45), reflexive sheaves in $\widetilde{M}_m$ form a dense open subset
$$ \begin{equation} \widetilde{R}_m:=\Theta(\widetilde{\mathcal R}_m)\cong\widetilde{\mathcal R}_m. \end{equation} \tag{4.51} $$

So the following theorem holds.

Theorem 4.4. Let $X=X_2$ be the quadric, and let $M_X(v)$ be the Gieseker–Maruyama moduli scheme of semistable sheaves $E$ on $X$ with Chern character $v=\mathrm{ch}(E)$ defined by (2.6) and (4.2). Then the following statements are true.

(i) The family

$$ \begin{equation} \widetilde{M}_m=\{[E]\in M_X(v)\mid E\textit{ is a stable extension } (4.2)\} \end{equation} \tag{4.52} $$
is a smooth dense open subset of an irreducible rational component of the scheme $M_X(v)$, and there is an isomorphism (4.50) in which $\widetilde{\mathcal{Y}}_m$ is the open subset, defined in (4.43), of the Grassmannization $\widetilde{\pi}\colon {\mathcal{G}}r(\widetilde{\mathcal{A}}^{\vee},2)\to\mathbb{G}$, where $\widetilde{\mathcal{A}}$ is the locally free sheaf on $\mathbb{G}$ given by the second formula in (4.15); moreover, $\widetilde{M}_m$ is a fine moduli space, and the reflexive sheaves form the dense open set $\widetilde{R}_m$ in $\widetilde{M}_m$ defined in (4.51).

(ii) The dimension of the variety $\widetilde{M}_m$ is given by the formula

$$ \begin{equation*} \begin{aligned} \, \dim\widetilde{M}_m &=2\binom{-m+4}{4}-2\binom{-m+2}{4}-2\binom{-m+3}{4} +2\binom{-m+1}{4}-2m+2 \\ &=2m^2-6m+4. \end{aligned} \end{equation*} \notag $$

Remark 4.3. On the quadric $X=X_2$ we consider a nontrivial extension of the form

$$ \begin{equation} 0\to\mathcal S(-1)\to E\to\mathcal I(m)\to 0, \qquad m\leqslant-1, \end{equation} \tag{4.53} $$
where $\mathcal{S}$ is the spinor bundle on $X$ and the sheaf $\mathcal{I}$ is the same as in (4.2). As in the case of the sheaves $E$ in (4.2), we assume that the sheaf $E$ is stable. Let $M_X(v)$ be the Gieseker–Maruyama moduli scheme of semistable sheaves $E$ with Chern character $v=\mathrm{ch}(E)$ defined by (2.4), (2.7) and (4.53). From (4.53) it follows that $E$ satisfies (4.9), (4.12), (4.14), (4.49) and the diagram (4.48), in which the sheaf $\mathcal{O}_X(-n)^{\oplus2}$ is throughout replaced by $\mathcal{S}(-1)$ and the left vertical triple in (4.48) is replaced by the triple $\mathbf{k}\rightarrowtail\mathbf{k}\twoheadrightarrow0$. Correspondingly, in the notation of the second formula in (4.15), the following locally free sheaf is defined on $\mathbb{G}$:
$$ \begin{equation} \widetilde{\mathcal A}_{\mathcal S}=\mathcal E xt_{\widetilde{p}}^1(\mathbb{I}\otimes\mathcal O_{\mathbb{G}}\boxtimes \mathcal O_X(m),\mathcal O_{\mathbb{G}}\boxtimes\mathcal S(-1)). \end{equation} \tag{4.54} $$
Next, as in Remark 4.2, we consider the projectivization $\widetilde{\pi}\colon \mathbb{P}(\widetilde{\mathcal{A}}_{\mathcal{S}}^{\vee})\to\mathbb{G}$ and its open subsets $\widetilde{ \mathcal{Y}}_{\mathcal{S},m}$ and $\widetilde{\mathcal{R}}_{\mathcal{S},m}$ of stable and reflective sheaves, respectively. For them, by analogy with (4.39) and (4.40) we have the embeddings of dense open sets
$$ \begin{equation} \widetilde{\mathcal R}_{\mathcal S,m}\subset\widetilde{\mathcal Y}_{\mathcal S,m}\subset{\mathcal G}r(\widetilde{\mathcal A}_{\mathcal S} ^{\vee},2) \end{equation} \tag{4.55} $$
and an isomorphism
$$ \begin{equation} \Theta\colon \widetilde{\mathcal Y}_{\mathcal S,m}\xrightarrow{\cong} M_m:=\Theta(\widetilde{\mathcal Y}_{\mathcal S,m}), \qquad [\widetilde{\xi}]\mapsto[E_{\widetilde{\xi}}]. \end{equation} \tag{4.56} $$
Repeating the proof of Theorem 4.4 for the varieties indicated we obtain the following theorem.

Theorem 4.5. Let $X=X_2$ be a quadric. Then in the notation of Remark 4.3 the following statements are true.

(i) The family

$$ \begin{equation*} M_m=\{[E]\in M_X(v)\mid E\ \textit{is a stable extension } (4.53) \} \end{equation*} \notag $$
is a smooth dense open subset of an irreducible rational component of the scheme $M_X(v)$, and there is an isomorphism $\Theta$ in (4.56), where $\widetilde{\mathcal{Y}}_{\mathcal{S},m}$ is an open subset of the projectivization $\widetilde{\pi}\colon \mathbb{P}(\widetilde{\mathcal{A}}_{\mathcal{S}}^{\vee})\to \mathbb{G}$ defined in Remark 4.3, where $\widetilde{\mathcal{A}}_{\mathcal{S}}$ is the locally free sheaf of rank $\operatorname{rk}\widetilde{\mathcal{A}}_{\mathcal{S}}=2m^2-8m+7$ on $\mathbb{G}$ defined by (4.54); in this case $M_m$ is a fine moduli space, and reflexive sheaves form the dense open set $\widetilde{R}_{\mathcal{S},m}=\Theta(\widetilde{\mathcal{R}}_{\mathcal{S},m})$ in $M_m$, where $\widetilde{\mathcal{R}}_{\mathcal{S},m}$ is defined in Remark 4.3.

(ii) The dimension of the variety $M_m$ is given by the formula $\dim M_m=2m^2-8m+10$.

4.3. The third series

Let $k\geqslant1$, $n=\lfloor\frac k2\rfloor+1$ and $m+n<0$. Consider a sheaf $F$ of rank $2$ on $X$ and the variety $W$ that are defined as follows.

(I) In the case $X=X_1$ the sheaf $F$ is determined from the exact triple

$$ \begin{equation} 0\to\mathcal O_{\mathbb{P}^3}(-1)\to\mathcal O_{\mathbb{P}^3}^{\oplus 3}\to F\to 0. \end{equation} \tag{4.57} $$
As is known (see, for example, [15], Example 4.2.1, or [2], Remark 2), the moduli space $W$ of sheaves $F$ is isomorphic to $\mathbb{P}^3$, and the isomorphism $W\xrightarrow{\simeq}\mathbb{P}^3$ is given by the formula $[F]\mapsto\operatorname{Sing}(F)$.

(II) In the case when $X=X_4$ is a complete intersection of a general pencil of hyperquadrics in $\mathbb{P}^5$, let $\mathbb{P}^1\subset|\mathcal{O}_{\mathbb{P}^5}(2)|$ be the base of this pencil of quadrics, and let $\Gamma$ be the hyperelliptic curve of genus 2 defined as the double covering $\rho\colon \Gamma\to \mathbb{P}^1$ branched at the points corresponding to degenerate quadrics in the pencil. Let $\Gamma^*=\rho^{-1}({\mathbb{P}^1}^*)$, where ${\mathbb{P}^1}^*\subset\mathbb{P}^1$ is an open subset of nondegenerate quadrics in the pencil, and let $\Delta=\rho^{-1}(\mathbb{P}^1\setminus{\mathbb{P}^1}^*)$. Each point $y\in\Gamma^*$ corresponds to one of the two series of generating planes on a nondegenerate four-dimensional quadric $Q(y):=\rho(y)$, and this series corresponds to a spinor bundle $\mathcal{S}(y)$ of rank $2$ on $Q(y)$ such that $\det\mathcal{S}(y)=\mathcal{O}_{Q(y)}(1)$. In this case we set $F_y=\mathcal{S}(y)|_X$. It is known that $\mathcal{S}(y)$ is an arithmetically Cohen–Macaulay sheaf (briefly, an ACM-sheaf), that is, it satisfies the equalities

$$ \begin{equation} h^i(\mathcal S(y)(\ell))=0, \qquad \ell\in\mathbb{Z}, \quad i=1,2. \end{equation} \tag{4.58} $$
From this and the cohomology of the triple $0\to\mathcal{S}(y)(-2)\to\mathcal{S}(y) \to F_y\to0$ it follows that $F_y$ is also an ACM-sheaf. Now let $y\in\Delta$, so that the degenerate quadric $Q(y)$ is a cone with vertex at a point, say, $z(y)$, and the projection $\mu$: $Q(y)\setminus \{z(y)\}\to Q_y$ is well defined, where $Q_y$ is a smooth three-dimensional quadric. On $Q_y$ the spinor bundle $\mathcal{S}_{Q_y}$ with $\det\mathcal{S}_{Q_y}=\mathcal{O}_{Q_y}(1)$ is defined, and we set $F_y=\mu^*\mathcal{S}_{Q_y}|_X$. The sheaf $F_y$ is also an ACM-sheaf. In this case we set $W=\{[F_y]\mid y\in \Gamma\}\simeq \Gamma$.

(III) In the case $X=X_5$ the sheaf $F$ is defined as the restriction to $X$ of the tautological bundle on the Grassmannian $\operatorname{Gr}(2,5)$ twisted by $\mathcal{O}_X(1)$. The isomorphism class $[F]$ of $F$ is uniquely defined, and we assume that $W$ is a point $\{[F]\}$. Note that $F$ is also an ACM-sheaf.

Note that in all cases (I)–(III) the universal family $\mathbb{F}$ of sheaves $F$ on $W\times X$ is defined.

Consider the sheaf $E$ on $X$ obtained as a nontrivial extension

$$ \begin{equation} 0\to F(-n)\to E\to\mathcal O_S(m)\to0, \qquad S\in|\mathcal O_X(k)|, \quad k\geqslant1. \end{equation} \tag{4.59} $$
Further, in the case when $E$ is stable, by analogy with (4.10), using the definition and the above properties of the sheaf $F$, the exact triple $0\to\mathcal{O}_X(-k)\to\mathcal{O}_X\to \mathcal{O}_S\to0$ and the condition $m+n<0$ we have the equalities
$$ \begin{equation} \begin{gathered} \, \begin{aligned} \, \hom(E,E) &=\hom(F,F)=\hom(\mathcal O_S(m),\mathcal O_S(m)) =\hom(E,\mathcal O_S(m)) \\ &=\hom(F(-n),E)=1, \end{aligned} \\ \hom(F(-n),\mathcal O_S(m)) =\operatorname{ext}^2(\mathcal O_S(m), F(-n))=0. \end{gathered} \end{equation} \tag{4.60} $$

As above, let $\mathbb{P}^N=|\mathcal{O}_X(k)|$, let $\mathbb{S}\subset\mathbb{P}^N \times X$ be a universal family of surfaces of degree $k$ in $X$, and let $W\times X\xleftarrow{q}W\times\mathbb{P}^N \times X\xrightarrow{p}W\times\mathbb{P}^N$ be the projections. On $W\times\mathbb{P}^N$, by analogy with (4.15)(4.17), we have a locally free sheaf

$$ \begin{equation} \mathcal A=\mathcal E xt_p^1(\mathcal O_{W\times\mathbb{P}^N}\boxtimes\mathcal O_X(m)|_{W\times\mathbb{S}},\mathcal O_{\mathbb{P}^N} \boxtimes \mathcal O_X(-n)\otimes q^*\mathbb{F}), \end{equation} \tag{4.61} $$
the construction of which commutes with the base change:
$$ \begin{equation} \tau_{F,S}\colon \mathcal A\otimes\mathbf k_{\{F,S\}}\xrightarrow{\cong} \operatorname{Ext}^1(\mathcal O_S(m),F(-n)), \end{equation} \tag{4.62} $$
and we have the equality
$$ \begin{equation} \mathcal E xt_p^0(\mathcal O_{W\times\mathbb{P}^N}\boxtimes\mathcal O_X(m)|_{W\times\mathbb{S}},\mathcal O_{W\times\mathbb{P}^N} \boxtimes\mathcal O_X(-n)\otimes q^*\mathbb{F})=0. \end{equation} \tag{4.63} $$
Consider the varieties $Y=\mathbb{P}(\mathcal{A}^{\vee})$ and $X_Y=Y\times X$ with projections $Y\xrightarrow{\rho}\mathbb{P}^N$ and $Y\xleftarrow{p_Y}X_Y\xrightarrow{\rho_Y}W\times\mathbb{P}^N\times X$. From (4.62) and (4.63) and Corollary 4.5 in [20] it follows that there is a universal family of extensions
$$ \begin{equation*} 0\to p_Y^*\mathcal O_P(1)\otimes(\mathcal O_{W\times\mathbb{P}^N}\boxtimes\mathcal O_X(-n)\otimes q^*\mathbb{F})\to\mathcal V \to\rho_Y^*(\mathcal O_{W\times\mathbb{P}^N}\boxtimes\mathcal O_X(m)|_{W\times\mathbb{S}})\to0 \end{equation*} \notag $$
on $X_Y$. Applying the functor $\rho_{Y*}$ to this triple we obtain a universal extension on $W\times\mathbb{P}^N\times X$, where $\mathcal{U}=\rho_{Y*}\mathcal{V}$:
$$ \begin{equation} 0\to p^*\mathcal A^{\vee}\otimes(\mathcal O_{W\times\mathbb{P}^N}\boxtimes\mathcal O_X(-n)) \to\mathcal U\to\mathcal O_{\mathbb{P}^N}\boxtimes\mathcal O_X(m)|_{\mathbb{S}}\to0. \end{equation} \tag{4.64} $$

Note that the extension (4.59) is specified by an element $\xi\!\in\!\operatorname{Ext}^1(\mathcal{O}_S(m), F(-n))$ considered as a homomorphism

$$ \begin{equation} \xi\colon \operatorname{Ext}^1(\mathcal O_S(m),\mathcal O_X(-n))^{\vee}\to\mathbf k. \end{equation} \tag{4.65} $$
By universality the exact triple (4.64), as restricted to $\{S\}\times X$, can in view of the base change (4.62) be included, together with the extension (4.59), in the commutative diagram
$(4.66)$
The homomorphism $\xi$ in (4.65) is nonzero since the extension (4.59) is nontrivial. Thus, $\xi$ is an epimorphism. The class $[\xi]$ of $\xi$ modulo the automorphisms of the space $\mathbf{k}$ is a point of the projectivization $\mathbb{P}(\operatorname{Ext}^1(\mathcal{O}_S(m),\mathcal{O}_X(-n))^{\vee})$. We have the subset $\mathcal{Y}_{k,m,n}$ in $\mathbb{P}(\mathcal{A}^{\vee})$ defined by
$$ \begin{equation*} \mathcal{Y}_{k,m,n}:=\{[\xi]\in\mathbb{P}(\mathcal{A}^{\vee})\mid \text{the extension}\ E_{[\xi]}\text{ in }(4.66)\text{ is stable}\}, \end{equation*} \notag $$
and the modular morphism $\Theta\colon \mathcal{Y}_{k,m,n}\to M_X(v)$, $[\xi]\mapsto[E_{[\xi]}]$ defined verbatim as in (4.29). At the same time, as in the case of the extension (4.1), by (4.60) the extension (4.59) defined by the sheaf $E$ is unique, that is, the class $[\xi]$ can be recovered from the point $[E]$. Thus, $\Theta$ is a bijection of the variety $\mathcal{Y}_{k, m,n}$ onto its image $M_{k,m,n}$ in $M_X(v)$, and by analogy with (4.31) we have the equalities
$$ \begin{equation} \begin{aligned} \, \notag \dim M_{k,m,n} &=\dim\mathcal Y_{k,m,n}=\dim\mathbb{P}(\operatorname{Ext}^1(\mathcal O_S (m),F(-n))^{\vee})+\dim W \\ &\qquad +\dim|\mathcal O_X(k)|=\operatorname{ext}^1(\mathcal O_S(m),F(-n))+\dim W+N-1. \end{aligned} \end{equation} \tag{4.67} $$
Applying the bifunctor $\mathbf{R}\operatorname{Hom}(-,-)$ to the triple (4.59) and taking (4.60) into account, we obtain a commutative diagram:
From this diagram and the equalities $\mathrm{ext}^1(\mathcal{O}_S(m),\mathcal{O}_S(m))=N$ and $\operatorname{ext}^1(F(-n),F(-n))=\dim W$ it follows that $\mathrm{ext}^1(E,E)=\mathrm{ext}^1(\mathcal{O}_S(m),F(-n))-1 +\dim W +N$, from which we find by (4.67) that $\dim\Theta(\mathcal{Y}_{k,m,n})=\mathrm{ext}^1(E,E) =\dim_{[E]} M_X(v)$. This means that $M_{k,m,n }$ is a smooth component of the moduli scheme $M_X(v)$. From this, as in (4.34) and (4.50), we obtain an isomorphism $\Theta\colon \mathcal{Y}_{k,m,n}\to M_{k,m,n}$, so that, $M_{k,m,n}$ is a smooth variety. Moreover, from the diagram (4.20) we obtain that $M_{k,m,n}$ is a fine moduli space. Precise formulae for the dimensions of the varieties $M_{k,m,n}$ are obtained using (4.67) by analogy with formulae (4.10). Thus, the following theorem holds.

Theorem 4.6. Let $X$ be one of varieties $X_1$, $X_4$ and $X_5$, let $k\geqslant1$, $n=\lfloor \frac k2\rfloor+1$, $m<-n$, and let $M_X(v)$ be the Gieseker–Maruyama moduli scheme of semistable sheaves $E$ on $X$ with Chern character $v=\mathrm{ch}(E)$ determined from the triple (4.59). Then

(i) the family

$$ \begin{equation*} \begin{aligned} \, M_{k,m,n} &=\{[E]\in M_X(v)\mid E \textit{ is specified by the extension } (4.59)\\ &\qquad\textit{in which $S\in|\mathcal O_X(k)|$, and the sheaf}\ F\ \textit{is determined} \\ &\qquad\textit{by the above conditions } \textrm{(I)}{-}\textrm{(III)}\} \end{aligned} \end{equation*} \notag $$
is a smooth irreducible component of the scheme $M_X(v)$, and it is described as the projectivization $\pi\colon \mathbb{P}(\mathcal{A}^{\vee})\to W\times\mathbb{P}^N$ of the bundle $\mathcal{A}^{\vee}$, where the variety $W$ is defined by conditions (I)–(III) above, and $\mathcal{A}$ is the locally free sheaf on $W\times\mathbb{P}^N$ defined by (4.61), where $\mathbb{P}^N=|\mathcal{O}_X(k)|$; this component is a rational variety for $X=X_1$, $X_2$ and $X_5$, and it is not rational for $X=X_4$; moreover, $M_{k,m,n}$ is a fine moduli space, and all sheaves in $M_{k,m,n}$ are stable;

(ii) the dimension of the variety $M_{k,m,n}$ is given by the following formulae:

    • (ii.1) for $X=X_1$,
      $$ \begin{equation*} \begin{aligned} \, \dim M_{k,m,n} &=\binom{k+3}{3}+3\binom{-m-n+k+3}{3}-\binom{-m-n+k+2}{3} \\ &\qquad -3\binom{-m-n+3}{3}+\binom{-m-n+2}{3}+1; \end{aligned} \end{equation*} \notag $$
    • (ii.2) for $X=X_4$,
      $$ \begin{equation*} \begin{aligned} \, \dim M_{k,m,n} &=4\biggl[\binom{k-m-n+4}{4}-\binom{k-m-n+2}{4} \\ &\qquad -\binom{-m-n+4}{4} +\binom{-m-n+2}{4}\biggr]; \end{aligned} \end{equation*} \notag $$
    • (ii.3) for $X=X_5$,
$$ \begin{equation*} \begin{aligned} \, \dim M_{k,m,n} &=\frac 53(k-m-n+1)^3+\frac 52(k-m-n+1)^2+\frac 56(k-m-n+1) \\ &\qquad-\frac 53(-m-n+1)^3-\frac 52(-m-n+1)^2 -\frac 56(-m-n+1) \\ &\qquad +\frac{k^3}{6}+\frac{k^2}{2}+ \frac{8}{15}k-1. \end{aligned} \end{equation*} \notag $$

4.4. Reflexive stable sheaves of general type with $c_1=0$ on $X_4$ and $X_5$

To conclude this section, we prove a theorem concerning reflexive sheaves in components of the Gieseker–Maruyama scheme constructed in Theorems 4.24.6.

For $X=X_4$ or $X_5$ we denote by $B(X)$ the base of the family of lines on $X$. As is known, $B(X_4)$ is a smooth abelian surface, and $B(X_5)\simeq\mathbb{P}^2$. Let us give the following definition, which will be key for us in § 6.

Definition 4.1. A reflexive sheaf $E$ of rank $2$ with first Chern class $c_1(E)=0$ on $X=X_4$ or $X=X_5$ is called a sheaf of general type if for any line $l\in B(X)$ not passing through points in $\operatorname{Sing} E$ we have either $E|_l\cong\mathcal{O}_{\mathbb{P}^1}^{\oplus2}$, and such lines form a dense open set in $B(X)$, or $E|_l\cong\mathcal{O}_{\mathbb{P}^1}(m) \oplus\mathcal{O}_{ \mathbb{P}^1}(-m)$, where $m>0$, and the set $B_2(X):=\{l\in B(X)\mid E|_l \cong\mathcal{O}_{\mathbb{P}^1}(m)\oplus\mathcal{O}_{\mathbb{P}^1}(-m),\ m\geqslant2\}$ has dimension ${\leqslant0}$.

The following theorem provides examples of infinite series of irreducible components of Gieseker–Maruyama moduli schemes for the varieties $X_4$ and $X_5$ such that general points of these components are reflexive sheaves of general type.

Theorem 4.7. Consider the following infinite series of components of the Gieseker–Maruyama moduli schemes of rank $2$ semistable sheaves on varieties $X_4$ and $X_5$:

Then in each of these components the general sheaf is a reflexive stable sheaf of general type.

Proof. (I) For $[E]\in M_{2,m,1}$ we have the triple (4.1) for $k=2$, $m\leqslant-2$ and $n=1$ and $S\in\mathbb{P}^N=|\mathcal{O}_X(2)|$:
$$ \begin{equation} 0\to\mathcal O_X(-1)^{\oplus 2}\to E\to\mathcal O_S(m)\to 0, \qquad S\in|\mathcal O_X(2)|. \end{equation} \tag{4.68} $$
Consider the variety $\Pi:=\{(\mathbb{P}^1,S)\in B(X)\times\mathbb{P}^N\mid \mathbb{P}^1\subset S\}$ with projections $B(X)\xleftarrow{\mathrm{pr}_1}\Pi\xrightarrow{\mathrm{pr}_2}\mathbb{P}^N$. An easy calculation shows that the dimension of a fibre of the projection $\mathrm{pr}_1$ is equal to $N-3$. On the other hand it is known [1] that $\dim B(X)=2$ for $X=X_4$. Hence $\dim\Pi=N-1<N=\dim\mathbb{P}^N$. Therefore, let us take a surface $S$ in the triple (4.68) such that $S\in\mathbb{P}^N\setminus(\mathrm{pr}_2(\Pi))$. By the definition of $\Pi$ this means that there are no projective lines on the surface $S$. Thus, taking an arbitrary projective line $\mathbb{P}^1\subset X$ disjoint from $\operatorname{Sing} E$, we have a scheme $Z_2:=S\cap\mathbb{P}^1$ of length $\ell(Z_2) =2$. Restricting the triple (4.68) to $\mathbb{P}^1$ we obtain the exact triple
$$ \begin{equation} 0\to\mathcal O_{\mathbb{P}^1}(-1)^{\oplus 2}\to E|_{\mathbb{P}^1}\to\mathcal O_{Z_2}\to0. \end{equation} \tag{4.69} $$
This restriction can be considered as a homomorphism of spaces of extensions $r\colon \operatorname{Ext}^1(\mathcal{O}_S(m),\mathcal{O}_X(-1)^{\oplus2})) \to\operatorname{Ext}^1(\mathcal{O}_{Z_2},\mathcal{O}_{\mathbb{P}^1}(-1)^{\oplus2}))$ for which $r(\xi)=\xi_{\mathbb{P}^1}$, where the elements $\xi$ and $\xi_{\mathbb{P}^1}$ define the extensions (4.68) and (4.69), respectively. The homomorphism $r$ factors into a composition
$$ \begin{equation*} \begin{aligned} \, r\colon \operatorname{Ext}^1(\mathcal O_S(m),\mathcal O_X(-n)^{\oplus2}) &\xrightarrow[\cong]{\varphi}H^0( \mathcal O_S(1-m)^{\oplus2})\xrightarrow{\otimes\mathcal O_{\mathbb{P}^1}}H^0(\mathcal O_{Z_2}^{\oplus2}) \\ &\xrightarrow[\cong]{\varphi_{\mathbb{P}^1}^{-1}}\operatorname{Ext}^1(\mathcal O_{Z_2},\mathcal O_{\mathbb{P}^1}(-1)^{\oplus2}), \end{aligned} \end{equation*} \notag $$
where $\varphi$ is an isomorphism (4.26) for $k=2$ and $n=1$, and the isomorphism $\varphi_{\mathbb{P}^1}$ is constructed by analogy with $\varphi$. Let us check that $r$ is an epimorphism. To do this it is sufficient to verify that the homomorphism $\otimes\mathcal{O}_{\mathbb{P}^1}$ is epimorphic. Consider the embedding $X=X_q\hookrightarrow\mathbb{P}^{q+1}$, $q\in\{4,5\}$, and let $L\cong\mathbb{P}^{q-1}$ be a subspace of codimension $2$ in $\mathbb{P}^{q+1}$ containing the subscheme $Z_2$ and intersecting $S$ along some zero-dimensional scheme $Z$. (A simple calculation of parameters shows that for a general subspace $L$ passing through $Z_2$ the condition $\dim Z=0$ is satisfied.) Then we have the $\mathcal{O}_S$-Koszul resolution of the sheaf
$$ \begin{equation*} \mathcal O_Z^{\oplus2}\colon 0\to\mathcal O_S(-m-1)^{\oplus2}\to\mathcal O_S(-m)^{\oplus4}\to\mathcal O_S(-m+1)^{\oplus2} \xrightarrow{\otimes\mathcal O_Z}\mathcal O_Z^{\oplus2}\to0. \end{equation*} \notag $$
Using the triple (4.7) and the condition $m\leqslant-2$, we find that $H^2(\mathcal{O}_S(-m-1))=H^1(\mathcal{O}_S(-m))=0$, and the resolution gives us an epimorphism $h^0(\varepsilon)\colon H^0(\mathcal{O}_S(-m+1)^{\oplus2})\to H^0(\mathcal{O}_Z^{\oplus2})$. Thus,
$$ \begin{equation*} \otimes\mathcal O_{\mathbb{P}^1}\colon H^0(\mathcal O_S(-m+1)^{\oplus2})\xrightarrow{h^0(\otimes\mathcal O_Z)}H^0( \mathcal O_Z^{\oplus2})\overset{h^0(\otimes\mathcal O_{Z_2})}{\twoheadrightarrow}H^0(\mathcal O_{Z_2}^{\oplus2}) \end{equation*} \notag $$
is an epimorphism.

Since $\mathbb{P}^1\cap\operatorname{Sing} E=\varnothing$, the sheaf $E|_{\mathbb{P}^1}$ is locally free, so the triple (4.69) shows that $E|_{\mathbb{P}^1}\cong\mathcal{O}_{\mathbb{P}^1}(a) \oplus\mathcal{O}_{\mathbb{P}^1}(-a)$, $0\leqslant a\leqslant1$. On the other hand, for general $\xi_{\mathbb{P}^1}\in\operatorname{Ext}^1(\mathcal{O}_{Z_2}, \mathcal{O}_{\mathbb{P}^1}(-1)^{\oplus2}))$ in (4.69) we have $E|_{\mathbb{P}^1}\cong\mathcal{O}_{\mathbb{P}^1}^{\oplus2}$. From this it follows in view of the surjectivity of $r$ that for general $\xi\in\operatorname{Ext}^1(\mathcal{O}_S(m),\mathcal{O}_X(-1)^{\oplus2}))$ the sheaf $E$ in (4.68) is a sheaf of general type.

(II) Consider the case $[E]\in M_{1,m,1}$ from Theorem 4.6. We have the triple (4.1) for $k=n=1$, $m\leqslant-1$ and $S\in\mathbb{P}^N=|\mathcal{O}_X(1)|$:

$$ \begin{equation} 0\to F(-1)\to E\to\mathcal O_S(m)\to 0, \qquad S\in|\mathcal O_X(1)|. \end{equation} \tag{4.70} $$
In this case for both the varieties $X=X_4$ and $X=X_5$ a general surface $S\in|\mathcal{O}_X(1)|$ is a smooth del Pezzo surface containing a finite number of lines and satisfying $S\cap\operatorname{Sing} E=\varnothing$. Then the sheaf $E|_{\mathbb{P}^1}$ is locally free, and the restriction of the triple (4.70) to any line $\mathbb{P}^1\subset S$ gives an epimorphism $E|_{\mathbb{P}^1}\twoheadrightarrow\mathcal{O}_{\mathbb{P}^1}(m)$, hence $E|_{\mathbb{P}^1}\cong\mathcal{O}_{\mathbb{P}^1}(a) \oplus\mathcal{O}_{\mathbb{P}^1}(-a)$, $a\leqslant m$. Now let $\mathbb{P}^1$ be a line on $X$ such that $\mathbb{P}^1\not \subset S$ and $\mathbb{P}^1\cap\operatorname{Sing} E=\varnothing$. Then, as before, the sheaf $E|_{\mathbb{P}^1}$ is locally free, and restricting the triple (4.70) to $\mathbb{P}^1$ we obtain the exact triple
$$ \begin{equation*} 0\to\mathcal O_{\mathbb{P}^1}\oplus\mathcal O_{\mathbb{P}^1}(-1)\to E|_{\mathbb{P}^1}\to\mathbf k_x\to0, \qquad x=\mathbb{P}^1\cap S. \end{equation*} \notag $$
By analogy with case (I), it suffices to check the surjectivity of the homomorphism $r\colon\operatorname{Ext}^1(\mathcal{O}_S(m),\mathcal{S}(-1)) \to\operatorname{Ext}^1(\mathbf{k}_x,\mathcal{O}_{\mathbb{P}^1} \oplus\mathcal{O}_{\mathbb{P}^1}(-1))$, which is in its turn equivalent to the surjectivity of the homomorphism $h^0(\otimes\mathcal{O}_Z)\colon H^0(\mathcal{S}(-m)|_S)\to H^0(\mathcal{O}_Z^{\oplus2})$ for a zero-dimensional scheme $Z=L\cap S$ containing the point $x$, where, as above, $L$ is a subspace of codimension $2$ of $\mathbb{P}^{q+1}$. The surjectivity of $h^0(\otimes\mathcal{O}_Z)$ for $m\leqslant-1$ is checked using the resolution
$$ \begin{equation*} 0\to F(-m-2)|_S\to F(-m-1)^{\oplus2}|_S\to F(-m)|_S\xrightarrow{\otimes\mathcal O_Z}\mathcal O_Z^{\oplus2}\to0. \end{equation*} \notag $$

Note that the reflexivity and stability of the general sheaf $E$ in (4.68) and (4.70) follow from Theorem 4.1.

Theorem 4.7 is proved.

§ 5. Moduli spaces of rank $2$ semistable sheaves with maximal third Chern class on the three-dimensional quadric $X_2$

5.1. The general case of rank $2$ semistable sheaves with $-1\leqslant c_1\leqslant0$ and maximal class $c_3$ on the quadric $X_2$

Stable sheaves $E$ of rank $2$ with maximal third Chern class, described in Theorem 3.1, are special cases of sheaves included in the exact triples (4.1), (4.2), (4.37) and (4.53). Namely, for a sheaf $E$ with $v=\operatorname{ch}(E)=(2,cH,dH^2,e[\mathrm{pt}])$ we have the following.

(1) For $c=-1$, a noninteger $d\leqslant-\frac32$ and $e=d^2-2d+\frac5{12}$, according to statement (1.4) of Theorem 3.1, the sheaf $E$ is included in the exact triple (4.1) for $k=n=1$, $m=d-\frac12$ and $S=Q_2$.

(2) For $c=-1$, an integer $d\leqslant-1$ and $e=d^2-2d+\frac16$, according to statement (1.3) of Theorem 3.1, the sheaf $E$ is included in the exact triple (4.2) for $m=d$ and $S=Q_2$.

(3) For $c=0$, a noninteger $d\leqslant-\frac32$ and $e=d^2+\frac14$, according to statement (2.4) of Theorem 3.1, the sheaf $E$ is included in the exact triple (4.37) for $k=n=1$, $m=d+\frac12$ and $S=Q_2$.

(4) For $c=0$, an integer $d\leqslant-3$ and $e=d^2$, according to statement (2.6) of Theorem 3.1, the sheaf $E$ is included in the exact triple (4.53) for $m=d$ and $S=Q_2$.

In this section we prove the following theorem, which gives a full description of the moduli spaces of rank $2$ semistable sheaves with maximal third Chern class on the quadric $X_2$ for all possible values of the second Chern class $c_2$ in cases (1)–(4) above.

Theorem 5.1. Let $X=X_2$ be a quadric and $M_X(v)$ be the Gieseker–Maruyama moduli scheme of rank $2$ semistable sheaves $E$ on $X$ with Chern classes $(c_1,c_2,c_3)$, where $c_1\in\{-1,0\}$, $c_2\geqslant0$ and $c_3=c_{3\max}$ is maximal for these $c_1$ and $c_2$, and let

$$ \begin{equation*} v=\operatorname{ch}(E)=\biggl(2,c_1H,\frac12(c_1^2-c_2)H^2, \frac12\biggl(c_{3\max}+\frac23c_1^3-c_1c_2\biggr) [\mathrm{pt}]\biggr) \end{equation*} \notag $$
according to (2.6). Then the scheme $M_X(v)$ is an irreducible and smooth rational projective variety, all sheaves in $M_X(v)$ are stable, a general sheaf in $M_X(v)$ is reflexive, and $M_X(v)$ is a fine moduli space. Moreover, the following statements hold.

(i) For $c_1=-1$, an even $c_2=2p$, $p\geqslant2$, and $c_{3\max}=\frac12c_2^2$ the variety $M_X(v)$ is the Grassmannization of the two-dimensional quotient spaces of the vector bundle $\mathcal{A}^{\vee}$ of rank $\operatorname{rk}\mathcal{A}=\frac14(c_2+2)^2$ on the space $\mathbb{P}^4$, which is defined by the first formula in (4.15) for $n=1$ and $m=-p$. In this case $\dim M_X(v)= \frac12(c_2+2)^2$.

(ii) For $c_1=-1$, an odd $c_2=2p+1$, $p\geqslant1$, and $c_{3\max}=\frac12(c_2^2-1)$ the variety $M_X(v)$ is the Grassmannization of two-dimensional quotient spaces of the vector bundle $\widetilde{\mathcal{A}}^{\vee}$ of rank $\operatorname{rk}\widetilde{\mathcal{A}}^{\vee}=\frac14(c_2+1)(c_2+3)$ on the Grassmannian $\mathbb{G}$ which is defined by the second formula in (4.15) for $m=-p$. In this case $\dim M_X(v)=\frac12(c_2+1)(c_2+3)$.

(iii) For $c_1=0$, an odd $c_2=2p+1$, $p\geqslant1$, and $c_{3\max}=\frac12(c_2^2+1)$ the variety $M_X(v)$ is the projectivization of the vector bundle $\mathcal{A}_{\mathcal{S}}^{\vee}$ of rank $\operatorname{rk}\mathcal{A}_{\mathcal{S}}^{\vee}=\frac12({c_2+1})({c_2+3})$ on the space $\mathbb{P}^4$ which is defined by formula (4.38) for $n=1$ and $m=-p$. In this case $\dim M_X(v)= \frac12c_2^2+2c_2+\frac92$.

(iv) For $c_1=0$, an even $c_2=2p$, $p\geqslant3$, and $c_{3\max}=\frac12c_2^2$ the variety $M_X(v)$ is the projectivization of the vector bundle $\widetilde{\mathcal{A}}_{\mathcal{S}}^{\vee}$ of rank $\operatorname{rk}\widetilde{\mathcal{A}}_{\mathcal{S}}^{\vee}= \frac12c_2^2+2c_2+1$ on the Grassmannian $\mathbb{G}$, which is defined by the formula (4.54) for $n=1$ and $m=1-p$. In this case $\dim M_X(v)=\frac12c_2^2+2c_2+4$.

Proof. (i), (ii) Let us show that each sheaf $E$ defined by the extension (4.1) for $k=n=1$, $m=-p$ and $S\in|\mathcal{O}_X(1)|$:
$$ \begin{equation} 0\to\mathcal O_X(-1)^{\oplus 2}\to E\to\mathcal O_{S}(-p)\to0, \qquad p\geqslant2, \end{equation} \tag{5.1} $$
or by the extension (4.2) for $m=-p$:
$$ \begin{equation} 0\to\mathcal O_X(-1)^{\oplus 2}\to E\to\mathcal J_{\mathbb{P}^1,S}(-p)\to0, \qquad p\geqslant1, \end{equation} \tag{5.2} $$
is stable, that is, that, in accordance with the definitions (4.22) and (4.43),
$$ \begin{equation} \mathcal Y_{1,m,1}={\mathcal G}r(\mathcal A^{\vee},2), \quad m\leqslant-2, \quad\text{and}\quad\widetilde{\mathcal Y}_m={\mathcal G}r(\widetilde{\mathcal A}^{\vee},2), \quad m\leqslant-1. \end{equation} \tag{5.3} $$
Indeed, specifying an extension (5.1) is equivalent to specifying an element of the space $\operatorname{Ext}^1(\mathcal{O}_{S}(-p),\mathcal{O}_X(-1)^{\oplus2})$. For a semistable sheaf $E$ the triple (5.1) gives its tilt Harder–Narasimhan filtration in the sense of Lemma 3.2.4 in [7], provided that $E$ is tilt-unstable. The Harder–Narasimhan tilt filtration factors are unique, that is, $E$ determines a surface $S$ and a subsheaf $\mathcal{O}_X(-1)^{\oplus 2}$. In this case the group $\mathrm{GL}(2)$ acts by automorphisms on the sheaf $\mathcal{O}_X(-1)^{\oplus 2}$ without changing the isomorphism class of the sheaf $E$. This means that there is a subspace of $\operatorname{Ext}^1(\mathcal{O}_{S}(-p),\mathcal{O}_X(-1))$. If this subspace is zero-dimensional, then $E$ is a direct sum, and therefore it is unstable. Assume that this subspace is one-dimensional. Then the morphism $\mathcal{O}_{S}(-p)\to\mathcal{O}_X(-1)^{\oplus2}[1]$ factors through $\mathcal{O}_X(-1)[1]$. Then the octahedron axiom implies the existence of a mapping $E\twoheadrightarrow\mathcal{O}_X(-1)$, contradicting the stability of $E$.

Conversely, let a two-dimensional subspace of $\operatorname{Ext}^1(\mathcal{O}_{S}(-p),\mathcal{O}_X(-1))$ be given. A choice of two basis vectors in this subspace gives an extension (5.1). Here $E$ is strictly semistable along the induced wall $W$. Its Jordan–Hölder filtration factors along the wall are two copies of the sheaf $\mathcal{O}_X(-1 )$ and one copy of the sheaf $\mathcal{O}_{S}(-p)$. Then the Jordan–Hölder factors of any destabilizing subobject for $E$ are among these sheaves. By construction $\operatorname{Hom}(E,\mathcal{O}_X(-1))=0$. Therefore, neither the sheaf $\mathcal{O}_{S}(-p)$ nor the extension of the sheaf $\mathcal{O}_{S}(-p)$ by $\mathcal{O}_X(-1)$ can be a subobject of $E$.

In the case of the extension (5.2) the reasoning is similar to the case of the extension (5.1). Now applying statements (i) and (ii.2) of Theorems 4.2 and 4.4 to the varieties $\mathcal{Y}_{1,m,1}={\mathcal{G}}r(\mathcal{A}^{\vee},2)$ and $\widetilde{\mathcal{Y}}_m={\mathcal{G}}r(\widetilde{\mathcal{A}}^{\vee},2)$ from (5.3), respectively, we obtain statements (i) and (ii) of the theorem.

(iii), (iv) Let us show that each sheaf $E$ defined by the extensions (4.37) and (4.53) for $k=n=1$, $m=-p$ and $S\in|\mathcal{O}_X(1)|$:

$$ \begin{equation} 0\to\mathcal S(-1)\to E\to\mathcal O_{S}(-p)\to0, \qquad p\geqslant1, \end{equation} \tag{5.4} $$
and
$$ \begin{equation} 0\to\mathcal S(-1)\to E\to\mathcal J_{\mathbb{P}^1,S}(-p)\to0, \qquad p\geqslant2, \end{equation} \tag{5.5} $$
is stable, that is, according to the definitions of $\mathcal{Y}_{\mathcal{S},1,m,1}$ and $\widetilde{\mathcal{Y}}_{\mathcal{S},1,m,1}$ (see Remarks 4.2 and 4.3, respectively), that
$$ \begin{equation} \mathcal Y_{\mathcal S,1,m,1}=\mathbb{P}(\mathcal A_{\mathcal S}^{\vee}), \quad m\leqslant-1, \quad\text{and}\quad \widetilde{\mathcal Y}_{\mathcal S,m}=\mathbb{P}(\widetilde{\mathcal A}_{\mathcal S}^{\vee}), \quad m\leqslant-2. \end{equation} \tag{5.6} $$
Indeed, an extension (5.4) corresponds to an element of $\operatorname{Ext}^1(\mathcal{O}_{S} (-p),\mathcal{S}(-1))$. For semistable objects this extension gives the tilt Harder–Narasimhan filtration for $E$, provided that $E$ is tilt-unstable. The Harder–Narasimhan factors in this case are unique. This means that $E$ determines a two-dimensional quadric $S$ and the sheaf $\mathcal{S}(-1)$ as a subobject of $E$. Multiplication of the morphism $\mathcal{S}(-1)\to E$ by a scalar does not change the isomorphism class of the sheaf $E$. In addition, if $E$ is a direct sum, then it is not stable.

Conversely, fix a one-dimensional subspace of $\operatorname{Ext}^1(\mathcal{O}_{S}(-p),\mathcal{S}(-1))$. The choice of any nonzero vector in this subspace gives a nontrivial extension (5.4). The object $E$ in it is strictly semistable along the wall $W$. The Jordan–Hölder factors of the object $E$ are uniquely defined, and the only subobjects that can destabilize $E$ above the wall are either $\mathcal{S}(-1)$ or $\mathcal{O}_{S}(-p)$. However, $\mathcal{S}(-1)$ does not destabilize $E$ for numerical reasons. On the other hand the nonsplitability of the previous exact triple means that $\mathcal{O}_{S}(-p)$ cannot be a subobject of $E$.

In the case of the extension (5.5) the argument is similar to the case of the extension (5.4). Now, applying statements (i) and (ii.2) of Theorems 4.3 and 4.5 to the varieties $\mathcal{Y}_{\mathcal{S},1,m,1}={\mathcal{G}}r(\mathcal{A}_{\mathcal{S}}^{\vee},2)$ and $\widetilde{\mathcal{Y}}_{\mathcal{S},1,m,1} ={\mathcal{G}}r(\widetilde{\mathcal{A}}_{\mathcal{S}}^{\vee},2)$ from (5.6), respectively, we obtain statements (iii) and (iv) of the theorem.

Theorem 5.1 is proved.

5.2. Special cases of rank $2$ semistable sheaves with small value of the class $c_2$ and the maximal class $c_3\geqslant0$ on $X_2$

Theorem 5.1 covers most possible values of the Chern classes of rank $2$ semistable sheaves $E$ on the quadric $X_2$. The remaining special cases for $c_3\geqslant0$ and small values of $c_2$ are covered by statements (1.1), (1.2), (2.1), (2.3) and (2.5) of Theorem 3.1. Below, in Theorem 5.2 we describe the moduli of sheaves $E$ for the first three of these cases, and in Theorems 5.3 and 5.4 we describe the ones for the last two cases, respectively.

Theorem 5.2. Under the assumptions and in the notation of Theorem 5.1 the following statements are true.

(1) For $c_1=-1$, $c_2=1$ and $c_{3\max}=0$ the variety $M_X(v)$ is the point $[\mathcal{S}(-1)]$.

(2) For $c_1=c_2=c_{3\max}=0$ the variety $M_X(v)$ is the point $[\mathcal{O}_X^{\oplus2}]$.

(3) For $c_1=-1$, $c_2=2$ and $c_{3\max}=2$ we have $M_X(v)\simeq \mathrm{Gr}(2,5)$.

Proof. Statements (1) and (2) follow directly from statements (1.1) and (2.1) of Theorem 3.1. Consider the last case.

(3) In this case, according to statement (1.2) of Theorem 3.1, the sheaf $E$ is determined by the exact triple

$$ \begin{equation} 0\to\mathcal O_X(-2)\xrightarrow{f}\mathcal O_X(-1)\otimes U\to E\to0. \end{equation} \tag{5.7} $$
The morphism $f$ can be considered as a three-dimensional quotient $U$ of the space $H^0(\mathcal{O}_X(1))^{\vee}$, that is, a point in the Grassmannian $\mathrm{Gr}(2,5)$. Consider the mapping $\varphi\colon\mathrm{Gr}(2,5)\to M\colon (H^0(\mathcal{O}_X(1))^{\vee} \twoheadrightarrow U)\mapsto E$, where $E$ is defined by (5.7).

Let us check that the function $\varphi$ is well defined. To do this, it is sufficient to verify that $E$ is $\mu$-stable. By Theorem 3.1 there is a unique wall $W$ for the above objects $E$, which is given by the exact triple

$$ \begin{equation} 0\to\mathcal O_X(-1)^{\oplus 3}\to E\to\mathcal O_X(-2)[1]\to0. \end{equation} \tag{5.8} $$
Therefore, checking that an object $E=\varphi(U)$ is $\mu$-stable is equivalent to checking the $\nu_{\alpha, \beta}$-stability of $E$ in a neighbourhood of this wall $W$ over it. If $E$ is not semistable over $W$, then there is a destabilizing semistable quotient $E\twoheadrightarrow G$. Since $E$ is strictly semistable along $W$, this quotient satisfies the equality $\nu_{\alpha,\beta}(E)=\nu_{\alpha, \beta}(G)$ for $(\alpha,\beta)$ on $W$. As is known, any Jordan–Hölder filtration for $E$ has three stable factors $\mathcal{O}_X(-1)$ and one stable factor $\mathcal{O}_X(-2)[1]$. Thus, the stable factors of the object $G$ are among these sheaves. The factor $\mathcal{O}_X(-2)[1]$ does not destabilize $E$ over $W$ for numerical reasons. The vector space $\operatorname{Hom}(E,\mathcal{O}_X(-1))$ is the kernel of the homomorphism $\operatorname{Hom}(\mathcal{O}_X(-1)\otimes U,\mathcal{O}_X(-1))\to\operatorname{Hom}(\mathcal{O}_X(-2), \mathcal{O}_X(-1))$, which is injective. Therefore, $G\not\cong\mathcal{O}_X(-1)^{\oplus a}$ for $a\in\{1,2,3\}$. If $G$ is an extension of $\mathcal{O}_X(-1)$ by $\mathcal{O}_X(-2)[1]$, then the corresponding subobject is $\mathcal{O}_X(-1)^{\oplus 2} $, and it does not destabilize $E$ above the wall. Likewise, if $G$ is an extension of $\mathcal{O}_X(-1)^{\oplus 2}$ by $\mathcal{O}(-2)[1]$, then the corresponding subobject is given by the sheaf $\mathcal{O}_X(-1)$, which does not destabilize $E$ above the wall. Thus, $E$ is $\mu$-stable.

Let us check that $\varphi$ is a bijection. According to Theorem 3.1, (1.2), any semistable object $E$ with $c_1=-1$, $c_2=2$ and $c_{3\max}=0$ satisfies the exact triple (5.8). Specifying such an extension is equivalent to specifying an element of $\operatorname{Ext}^1(\mathcal{O}_X(-2)[1],\mathcal{O}_X(-1)^{\oplus 3})=H^0(\mathcal{O}_X(1))^{\oplus 3}$. In Theorem 3.1 we showed that this extension is the tilt Harder–Narasimhan filtration for $E$ under the wall. The factors of this filtration are uniquely defined. This means that $E$ determines the subobject $\mathcal{O}_X(-1)^{\oplus 3}$. However, the group $\mathrm{GL}(3)$ acts on $\mathcal{O}_X(-1)^{\oplus 3}$ by automorphisms without changing the isomorphism class of $E$. This means that we obtain a unique subspace of $H^0(\mathcal{O}_X(1))$. However, if this subspace is not three-dimensional, then there is a destabilizing morphism $E\twoheadrightarrow\mathcal{O}_X(-1)$. This proves both the surjectivity and injectivity of $\varphi$.

Let us show that $\varphi$ is a morphism of schemes. To do this it is sufficient to construct a universal family $\mathcal{E}$ of sheaves $E$ on $\mathrm{Gr}(2,5)\times \mathbb{P}^3$. Then the universal property of the scheme $M_X(v)$ shows that $\varphi$ is a morphism. We have the two projections $\mathrm{Gr}(2,5)\xleftarrow{p_1}\mathrm{Gr}(2,5)\times X\xrightarrow{p_2}X$ and the rank $3$ tautological quotient bundle of $\mathcal{O}_{\mathrm{Gr}(2,5)}\otimes H^0(\mathcal{O}_X(1))^{\vee}\twoheadrightarrow\mathcal{Q}$ on $\mathrm{Gr}(2,5)$. The required universal family $\mathcal{E}$ is obtained as the cokernel of the composition of morphisms $p_2^*\mathcal{O}_X(-2)\to H^0(\mathcal{O}_X(1))^{\vee} \otimes p_2^*\mathcal{O}_X(-1)\to\mathcal{Q}\boxtimes\mathcal{O}_X(-1)$.

To prove the smoothness of $M_X(v)$ it is sufficient to show that $\operatorname{Ext}^2(E,E)=0$. This is done by applying the three functors ${\mathbf R}\operatorname{Hom}(-,\mathcal{O}_X(-2)[1])$, ${\mathbf R}(-,\mathcal{O}_X(-1))$ and ${\mathbf R}\operatorname{Hom}(E,-)$ to the exact triple (5.8). From the smoothness of $M_X(v)$ and the bijectivity of $\varphi$ and Theorem 2.16 in [28], Ch. 2, § 4.4, it follows that $\varphi$ is an isomorphism.

The theorem is proved.

Theorem 5.3. For $c_1=0$, $c_2=2$ and $c_{3\max}=2$ the scheme $M_X(v)$ is irreducible, has dimension $9$ and is not smooth.

Proof. Let $E$ be a Gieseker-semistable sheaf with $c_1=0$, $c_2=2$ and $c_{3\max}=2$, so that $[E]\in M_X(v)$, where $v=(2,0,-H^2, \{\mathrm{pt}\})$. By Theorem 3.1, (2.3), $E$ is destabilized by an exact triple
$$ \begin{equation} 0\to\mathcal S(-2)^{\oplus2}\xrightarrow{\alpha}\mathcal O_X(-1)^{\oplus 6}\to E\to 0. \end{equation} \tag{5.9} $$

Let us prove that there exist stable sheaves $E$ included in (5.9). Let $E$ be properly semistable. Then there exists a subsheaf $F\subset E$ of rank $1$ such that $\chi(F(m))=\frac 12\chi(E(m))$ for $m\gg 0$. From this we obtain $F\in M(1,0,-\frac12 H^2,\frac 12)$. Since $F$ is stable, by the proof of Lemma 3.4 we can find an exact triple $0\to\mathcal{S}(-2)\to\mathcal{O}_X(-1)^{\oplus3}\to F\to 0$. Since $\mathrm{Ext}^1 (\mathcal{O}_X(-1)^ {\oplus 6}, \mathcal{S}(-2)^{\oplus2})=0$, the embedding ${F\to E}$ induces a morphism $\mathcal{O}_X (-1)^{\oplus3}\to \mathcal{O}_X(-1)^{\oplus 6}$. Set $W=H^0(\mathcal{S}(1))\cong\mathbf{k}^{16}$ and $A=H^0(\alpha^\vee(-1))\colon\mathbf{k}^6\to\mathbf{k}^2\otimes W$; then we obtain the commutative diagram with exact rows

In other words, the matrix of the mapping $A$ has the following form in some bases of the spaces $\mathbf{k}^6$ and $\mathbf{k}^2$ (consistent with the choice of the subspaces $\mathbf{k}^3\subset\mathbf{k}^6$ and $\mathbf{k}\subset\mathbf{k}^2$) has the form
$$ \begin{equation*} \begin{pmatrix} * & * & * & * & * & * \\ 0 & 0 & 0 & * & * & * \end{pmatrix}. \end{equation*} \notag $$
Consider the incidence variety
$$ \begin{equation*} X=\{(A_i)_{i=1}^{16}\in\operatorname{Hom}(\mathbf k^6,\mathbf k^2)^{\oplus16},V_1\in\operatorname{Gr}(3,6), V_2\in\operatorname{Gr}(1,2)|A_i(V_1)\subset V_2\} \end{equation*} \notag $$
with the projection $X\to Y=\mathrm{Hom}(\mathbf{k}^6,\mathbf{k}^2)^{\oplus16}$. Because $\dim\mathrm{Gr}(3,6)=9$, $\dim\mathrm{Gr}(1,2)=1$ and three independent linear conditions are imposed on each of the 16 matrices $A_i$, $\dim X=16\cdot12+9+1-16\cdot3<16\cdot12=\dim Y$. It follows that there is a dense open subset of the variety $Y$ that does not intersect the image of $X$ under the projection $X\to Y$. This proves the existence of Gieseker-stable sheaves $E=\operatorname{coker}\alpha$.

Since $E$ is tilt-semistable over a semicircular wall, it follows that $\operatorname{Hom}(E,\mathcal{O}_X(-1))=0$. In addition, $\operatorname{Ext}^1(\mathcal{S}(-2),\mathcal{S}(-2))=0$ (since $\mathcal{S}(-1)$ is an exceptional object of the category $\mathrm{D}^b(X)$) and $\operatorname{Ext}^2(\mathcal{O}_X(-1),\mathcal{S}(-2))=0$ (since $\mathcal{S}(-1)$ is an ACM-sheaf). Now it follows from (5.9) that $\operatorname{Ext}^2(E, \mathcal{S}(-2)^{\oplus2})=0$, so we have an exact sequence

$$ \begin{equation} \begin{aligned} \, \notag 0 &\to\operatorname{Hom}(E,E)\to\operatorname{Ext}^1(E,\mathcal S(-2)^{\oplus 2}) \\ &\to\operatorname{Ext}^1(E,\mathcal O_X(-1)^{\oplus 6})\to\operatorname{Ext}^1(E,E)\to0. \end{aligned} \end{equation} \tag{5.10} $$
Now if the sheaf $E$ is stable, then $\hom(E,E)=1$, and calculation using (5.10) and (5.9) gives us $\operatorname{ext}^1(E,E)=9$. On the other hand, from the triple (5.9) it follows that $M_X(v)$ is irreducible and $\dim M_X(v)=12h^0(\mathcal{S})-\dim(\mathrm{GL}(2,\mathbf{k})\times \mathrm{GL}(6,\mathbf{k})/\mathbf{k}^*)=12\cdot4-4-36+1=9$, so $M_X(v)$ is smooth at the point $[E]$.

In the properly semistable case we use the following result.

Lemma 5.1 ([22], p. 294, the lemma). Let $E$ be a semistable sheaf of rank $2$, and let $E\cong E_1\oplus E_2$, where $\chi(E_1(m))= \chi(E_2(m))=\frac 12\chi(E(m))$ and $E_1\not\cong E_2$. If $\operatorname{Ext}^2(E,E)=0$, then the tangent space to the moduli scheme of semistable sheaves at the point $[E]$ is isomorphic to $\operatorname{Ext}^1(E_1,E_1)\oplus(\operatorname{Ext}^1(E_1,E_2) \oplus\operatorname{Ext}^1(E_2,E_1))\oplus\operatorname{Ext}^1(E_2,E_2)$.

In our case we apply this lemma to the sheaf $E\cong E_1\oplus E_2$, where $E_1\not\cong E_2$ and the sheaves $E_1$ and $E_2$ are stable and included in exact triples of the form

$$ \begin{equation} 0\to\mathcal S(-2)\to\mathcal O_X(-1)^{\oplus 3}\to E_i\to 0. \end{equation} \tag{5.11} $$
Since $\operatorname{Ext}^1(\mathcal{S}(-2),\mathcal{O}_X(-1)))\cong H^1(\mathcal{S})=0$ and $\operatorname{Ext}^2(\mathcal O_X(-1),\mathcal O_X(-1))=0$, taking (5.11) into account we have $\operatorname{Ext}^2(E,\mathcal{O}_X(-1)^{\oplus 6})=0$. On the other hand, since $\operatorname{Ext}^2(\mathcal{S}(-2),\mathcal{S}(-2)) =\operatorname{Ext}^3(\mathcal{O}_X(-1),\mathcal{S}(-2))=0$ and the pair $(\mathcal{S}(-1),\mathcal{O}_X)$ is exceptional in $\mathrm{D}^b(X)$, we have $\operatorname{Ext}^3(E,\mathcal{S}(-2))=0$. From this and (5.9) we obtain $\operatorname{Ext}^2(E,E)=0$. Now by Lemma 5.1 the dimension of the tangent space to $M_X(v)$ at the point $[E]=[E_1\oplus E_2]$ is equal to $3+2+2+3=10\neq 9$, so $M_X(v)$ is not smooth.

Theorem 5.3 is proved.

Theorem 5.4. For $c_1=0$, $c_2=4$ and $c_{3\max}=8$ the scheme $M_X(v)$ is a union of two irreducible components $M_1$ and $M_2$. These components are described as follows.

(i) $M_1$ is a smooth rational variety of dimension $20$, which is the projectivization of a locally free sheaf of rank $17$ on $\mathbb{G}$. $M_1$ is a fine moduli space, and all sheaves from $M_1$ are stable. The scheme $M_X(v)$ is nonsingular along $M_1$.

(ii) The scheme $M_2$ is irreducible, has dimension $21$, and polystable sheaves in $M_2$ form a closed subset of dimension $12$ of $M_2$, in which the scheme $M_X(v)$ is not smooth.

(iii) The scheme $M_X(v)$ is disconnected:

$$ \begin{equation*} M_X(v)=M_1\sqcup M_2. \end{equation*} \notag $$

Proof. Let $E$ be a Gieseker-semistable sheaf with $c_1=0$, $c_2=2$ and $c_{3\max} =2$, so that $[E]\in M_X(v)$, where $v=(2,0,-2H^2,4\{\mathrm{pt}\})$. By Theorem 3.1, (2.5), the sheaf $E$ is destabilized by an exact triple
$$ \begin{equation} 0\to\mathcal S(-1)\to E\to\mathcal I_{\mathbb{P}^1,S}(-1)\to0 \end{equation} \tag{5.12} $$
or an exact triple
$$ \begin{equation} 0\to\mathcal O_X(-2)^{\oplus2}\to\mathcal O_X(-1)^{\oplus 4}\to E\to0. \end{equation} \tag{5.13} $$

(i) According to Lemma 3.8, any sheaf $E$ in a triple (5.12) is tilt-stable for $\beta<0$ and $\alpha\gg0$, and therefore it is stable according to Proposition 2.1. In other words, in the notation of Remark 4.3 we have the equality $\widetilde{\mathcal{Y}}_{\mathcal{S},1,m,1} =\mathbb{P}(\widetilde{\mathcal{A}}_{\mathcal{S}}^{\vee})$, which is similar to the second equality in (5.6) (where $m=-1$ in our case). Therefore, applying Theorem 4.5 to $k=n=1$ and $m=-1$ we find that the bundles $E$ included in a triple (5.12) form a smooth irreducible rational component $M_1:=M_{1,-1,1}$ of the scheme $M_X(v)$, described as the projectivization of $\widetilde{\pi}\colon \mathbb{P}(\widetilde{\mathcal{A}}_{\mathcal{S}}^{\vee})\to\mathbb{G}$, where $\widetilde{\mathcal{A}}_{\mathcal{S}}$ is a locally free sheaf of rank $17$ on $\mathbb{G}$. In this case $\dim M_1=20$, $M_1$ is a fine moduli space, and, according to the above, all sheaves in $M_1$ are stable.

(ii) Set $M_2=\{[E]\in M_X(v)\mid E\text{ is included in a triple }(5.13)\}$. From (5.13) we obtain that $M_2$ is irreducible as the quotient of an open subset of the space $\operatorname{Hom}(\mathcal{O}_X(-2)^{\oplus2},\mathcal{O}_X(-1)^{\oplus4})$ by the group $\dim(\mathrm{GL}(2,\mathbf{k}) \times \mathrm{GL}(4,\mathbf{k})/\mathbf{k}^*$, and $\dim M_2=\hom(\mathcal{O}_X(-2)^{\oplus2},\mathcal{O}_X(-1)^{\oplus4}) -\dim(\mathrm{GL}(2,\mathbf{k})\times \mathrm{GL}(4,\mathbf{k})/\mathbf{k}^*)={2\cdot4\cdot5}-4-16+1=21$. Next, for sheaves $E$ in $M_2$ we carry out an argument similar to the argument for the triple (5.9) carried out in the proof of Theorem 5.3. In particular, we verify that $M_2$ contains stable sheaves $E$. For stable sheaves $E$ we have $\operatorname{Hom}(E,\mathcal O_X(-1)^{\oplus 4})=0$, and also $\operatorname{Ext}^2(E,\mathcal{O}_X(-2)^{\oplus2})=0$ because of the exceptionality of line bundles on $X$ and the ACM property. Thus, in this case there is an exact sequence similar to (5.10):

$$ \begin{equation*} 0\,{\to}\operatorname{Hom}(E,E)\to\operatorname{Ext}^1(E,\mathcal O_X(-2)^{\oplus2})\,{\to}\operatorname{Ext}^1(E,\mathcal O_X(-1)^{\oplus 4})\to\operatorname{Ext}^1 (E,E)\to0. \end{equation*} \notag $$
A standard calculation using (5.13) with this sequence shows that the dimension $\operatorname{Ext}^1(E,E)$ of the tangent space to $M_X(v)$ at the point $[E]$ is equal to $21$, which coincides with $\dim M_2$. Thus, $M_2$ is an irreducible component of the scheme $M_X(v)$.

Now consider a point $[E]=[E_1\oplus E_2]$, where $E_1\not\cong E_2$ and the $E_i$ are included in exact triples of the form

$$ \begin{equation} 0\to\mathcal O_X(-2)\to\mathcal O_X(-1)^{\oplus 2}\to E_i\to0, \qquad i=1,2. \end{equation} \tag{5.14} $$
A simple calculation using these triples shows that $\operatorname{Ext}^2(E_i,E_j)=0$, $1\leqslant i$, $j\leqslant2$, $\operatorname{ext}^1(E_i,E_i)=6$, $i=1,2$, and $\operatorname{ext}^1(E_i,E_j)=5$, $i\ne j$. Hence $\operatorname{Ext}^2(E,E) =0$, and by Lemma 5.1 $\dim T_{[E]}M_X(v)=6+5+5+6=22\neq 21$, and therefore $M_X(v)$ is not smooth at the point $[E]$.

(iii) According to the last statement of Lemma 3.8, $M_1\cap M_2=\varnothing$, that is, the union $M_1\cup M_2$ is disjoint.

Theorem 5.4 is proved.

§ 6. Boundedness of the third Chern class of rank $2$ stable reflexive sheaves of general type with $c_1=0$ on the varieties $X_4$ and $X_5$

In this section we consider rank $2$ stable reflexive sheaves of general type with $c_1=0$ (see Definition 4.1) on the varieties $X_4$ and $X_5$. For any such sheaf $E$ we prove that the third Chern class $c_3$ of the sheaf $E$ is bounded above by a quadratic polynomial in the second Chern class $c_2$: see Theorems 6.1 and 6.2 below.

6.1. Rank $2$ stable reflexive sheaves of general type with $c_1=0$ on the variety $X_5$

Throughout this subsection $X=X_5$.

Recall some well-known facts about the variety $X$ (see, for example, [1], Theorem 4.2, (iii), and Corollary 6.6, (ii)).

(1) The base $B=B(X)$ of the family of lines on $X$ is isomorphic to $\mathbb{P}^2$.

(2) For an arbitrary line $l\in B$ the set $B_l=\{l'\in B\mid l'\cap l\ne\varnothing\}$ is a line in $\mathbb{P}^2$.

(3) For an arbitrary line $l\in B$ either $N_{l/X}\cong\mathcal{O}_{\mathbb{P}^1}^{\oplus2}$ (general case) or $N_{l/X}\cong\mathcal{O}_{\mathbb{P}^1}(1)\oplus\mathcal{O}_{\mathbb{P}^1}(-1)$.

(4) Let $\varphi\colon X\dashrightarrow \mathbb{P}^4$ be the linear projection of the variety $X\subset\mathbb{P}^6$ onto the space $\mathbb{P}^4$ from a line $l\subset X$ for which $N_{l/X}\cong\mathcal{O}_{\mathbb{P}^1}^{\oplus2}$. Then $\varphi(X)=Q\cong X_2$ is a smooth quadric in $\mathbb{P}^4$, and the morphism $\varphi\colon X\dashrightarrow Q$ is birational and decomposes into a Hironaka roof:

$(6.1)$
Here $\delta^{-1}\colon X\dashrightarrow\widetilde{X}$ is the blow-up of $X$ centred at $l$, $S =\delta^{-1}(l)\simeq\mathbb{P}^1\times\mathbb{P}^1$, $\sigma|_S\colon S\xrightarrow{\simeq}\sigma(S) =Q_2$ is an isomorphism of $S$ onto a smooth two-dimensional quadric $Q_2\subset Q$, and $\sigma^{-1}\colon Q\dashrightarrow\widetilde{X}$ is the blow-up of $Q$ centred in a smooth rational cubic curve $C$ of type (2,1) on $Q_2$, so that $\widetilde{X}\simeq\mathbb{P}(\mathcal{I}_{C,Q})$. In particular, the triple $0\to N_{C,Q_2}\to N_{C/Q}\to N_{Q_2/Q}|_{C}\to0$ is exact, where in view of the isomorphisms $Q_2\simeq\mathbb{P}^1 \times\mathbb{P}^1$ and $C\simeq\mathbb{P}^1$ we obtain $N_{C/Q_2}\cong\mathcal{O}_{\mathbb{P}^1}(3)$ and $N_{Q_2/Q}|_{C}\cong\mathcal{O}_Q(H)|_{C}\cong\mathcal{O}_{\mathbb{P}^1}(3)$, hence $\det N_{C/ Q}\cong \mathcal{O}_{\mathbb{P}^1}(6)$.

(5) Let $\Gamma=\{(x,l)\in X\times B\mid x\in l\}$ be an incidence graph with projections $X\xleftarrow{p_1}\Gamma\xrightarrow{p_2}B$. As we know (see [1], Proposition 5.2), $p_1\colon\Gamma\to X$ is a morphism which is finite at a general point, and we have $\dim\{x\in X\mid \dim p_1^{-1} (x)\geqslant1\}\leqslant0$.

Consider an extension

$$ \begin{equation} \xi\colon 0\to\mathcal O_Q(nH)\to F\xrightarrow{\varepsilon}\mathcal I_{C,Q}\to0. \end{equation} \tag{6.2} $$
The sheaf $\mathcal{H}om(\mathcal{I}_{C,Q},\mathcal{O}_Q(nH))\cong\mathcal{O}_Q(nH)$ on $Q$ and the sheaf $\mathcal{M}=\mathrm{Ext}^1(\mathcal{I}_{C,Q},\mathcal{O}_Q(nH)) \cong\mathcal{E}xt^2(\mathcal{O}_C,\mathcal{O}_Q(nH))\cong\det N_{C/Q}\otimes\mathcal{O}_Q(nH)\cong\mathcal{O}_{\mathbb{P}^1}(6+3n)$ on $C$ are very ample for $n\gg0$. Therefore, we can assume the following.

(1) $H^i(\mathcal{H}om(\mathcal{I}_{C,Q},\mathcal{O}_Q(nH)))=0$, $i=1,2$, which implies that the long exact sequence of $\operatorname{Ext}$-groups for a pair of sheaves $\mathcal{I}_{C,Q},\mathcal{O}_Q(nH)$ gives the isomorphisms $\mathrm{Ext}^1(\mathcal{H}om(\mathcal{I}_{C,Q},\mathcal{O}_Q(nH)))\cong H^0(\mathrm{Ext}^1(\mathcal{I}_{C,Q},\mathcal{O}_Q(nH))) \cong H^0(\mathrm{Ext}^2(\mathcal{O}_C,\mathcal{O}_Q(nH)))=H^0(\mathcal{M})$.

(2) The triple (6.2) as an extension of $\xi\in\mathrm{Ext}^1(\mathcal{H}om(\mathcal{I}_{C,Q}, \mathcal{O}_Q(nH)))=H^0(\mathcal{M})$ is a section of the sheaf $\mathcal{M}$, whose scheme of zeros $(\xi)_0$ is reduced, that is, it is a simple divisor $D_{\xi}=x_1+\dots+x_{3n+6}$. By Serre’s construction this means (see [15], § 4) that $F$ in (6.2) is a reflexive sheaf of rank $2$ with $\operatorname{Sing} F=D_{\xi}$ and with the simplest singularities at the points $x_i$, that is, $\mathrm{Ext}^1(F,\mathcal{O}_Q)\cong\oplus_{i=1}^{3n+6}\mathbf{k}_{x_i}$. Thus, according to Corollary 2.8 in [4], the projective spectrum $Y=\mathbb{P}(F)$ of the sheaf $F$ is a smooth variety. The epimorphism $\varepsilon$ in (6.2) corresponds to an embedding $i$ of the divisor $\widetilde{X}=\mathbb{P}(\mathcal{I}_{C,Q})$ into $Y$, and let $p\colon Y\to Q$ be the natural projection, so that $\sigma=p\circ i$. Note also that the sheaf

$$ \begin{equation*} L=\mathcal O_{Y/Q}(1)\otimes p^*\mathcal O(mH) \end{equation*} \notag $$
is ample for $m\gg0$, where $\mathcal{O}_{Y/Q}(1)$ is the Grothendieck sheaf on $Y=\mathbb{P}(F)$.

Let

$$ \begin{equation} \begin{aligned} \, & E\text{ be a rank $2$ stable reflexive sheaf of general type} \\ &\text{with}\ c_1(E)=0 \ \text{on}\ X;\ \text{in particular,}\ H^0(E)=0. \end{aligned} \end{equation} \tag{6.3} $$

Since $\dim\operatorname{Sing} E\leqslant0$, it follows from properties (1)–(5) that we can choose for $l$ a general line on $X$ for which $N_{l/X}\cong\mathcal{O}_{\mathbb{P}^1}^{\oplus2}$ and such that the following conditions are satisfied.

(a) $B_l\cap p_2(p_1^{-1}(\operatorname{Sing} E))=\varnothing$, so the surface $S_l=p_1(p_2^{-1}(B_l))$ does not intersect the set $\operatorname{Sing} E$. Since by construction $S_l=\delta(S)$, this means that the sheaf

$$ \begin{equation} \widetilde{E}=\delta^*E \end{equation} \tag{6.4} $$
satisfies the condition
$$ \begin{equation} \operatorname{Sing}\widetilde{E}\cap S=\varnothing. \end{equation} \tag{6.5} $$
Note that it follows from the projection formula for the blow-up $\delta$ that $E=\delta_*\widetilde{E}$ and $R^i\delta_*\widetilde{E}=0$, $i>0$. Therefore,
$$ \begin{equation} \chi(E)=\chi(\widetilde{E}) \end{equation} \tag{6.6} $$
and, in addition, (6.3) implies that
$$ \begin{equation} H^0(\widetilde{E})=0. \end{equation} \tag{6.7} $$

(b) Since $E$ is a sheaf of general type, the set $G_C=\{x\in C\mid \widetilde{E}|_{\sigma^{-1}(x)}\cong\mathcal{O}_{\mathbb{P}^1}(1) \oplus\mathcal{O}_{\mathbb{P}^1}(-1)\}$ is a proper subset of the curve $C$, and $\widetilde{E}|_{\sigma^{-1}(x)}\cong\mathcal{O}_{\mathbb{P}^1}^{\oplus2}$ for $x\in C\setminus G_C$.

In addition, we assume that the section $\xi$ of the sheaf $\mathcal{M}$ is sufficiently general, so that

$$ \begin{equation} D_{\xi}\cap G_C=\varnothing. \end{equation} \tag{6.8} $$
Below we need another sufficiently general section $\xi'\in H^0(\mathcal{L})$ such that
$$ \begin{equation} D_{\xi'}\cap G_C=\varnothing \quad\text{and}\quad D_{\xi}\cap D_{\xi'} =\varnothing. \end{equation} \tag{6.9} $$

Put

$$ \begin{equation} \begin{gathered} \, Q^0:=Q\setminus(p(\operatorname{Sing}\widetilde{E})\cup D_{\xi}), \qquad C^0:=C\setminus D_{\xi}\hookrightarrow Q^0, \\ X^0:=\sigma^{-1}(Q^0)=\widetilde{X}\setminus (\operatorname{Sing}\widetilde{E}\cup\sigma^{-1}D_{\xi}), \\ Y^0=Y\setminus p^{-1}(Q^0), \qquad\widetilde{E}^0:=\widetilde{E}|_{X^0} \quad\text{and}\quad\overline{E}^0:=\sigma_*\widetilde{E}^0, \end{gathered} \end{equation} \tag{6.10} $$
where $X^0$ becomes a divisor in $Y^0$ via the embedding $i$ defined above, so that the following diagram is commutative:
$(6.11)$
Note that $\operatorname{Sing} F=D_{\xi}$ and $Q_0$ are disjoint by (6.10), so that the $\mathcal{O}_{Q^0}$-sheaf $F|_{Q^0}$ is locally free, and therefore $p\colon Y^0= \mathbb{P}(F|_{Q^0})\to Q^0$ is a locally trivial $\mathbb{P}^1$-bundle. Hence by Serre’s theorem, for sufficiently large natural numbers $m$, $s$ and $r$ there is an epimorphism $e\colon\mathcal{L}_0:=(L^{\otimes-s}|_{Y^0})^{\oplus r}\twoheadrightarrow \widetilde{E}^0$. Since $\overline{E}^0$ is a locally free $\mathcal{O}_{X^0}$-sheaf on a smooth divisor $X^0$ in a smooth variety $Y^0$, $\widetilde{E}^0$, considered as an $\mathcal{O}_{Y^0}$-sheaf, has homological dimension $1$, so that $\mathcal{L}_1:=\ker e$ is a locally free $\mathcal{O}_{Y^0} $-sheaf. Thus, there is an exact triple
$$ \begin{equation} 0\to\mathcal L_1\to\mathcal L_0\xrightarrow{e}\widetilde{E}^0\to0. \end{equation} \tag{6.12} $$
From the definition (6.10) of the curve $C^0$ it follows that
$$ \begin{equation} \widetilde{E}^0|_{\sigma^{-1}(x)}\cong \begin{cases} \mathcal O_{\mathbb{P}^1}^{\oplus2}, &x\in C^0\setminus G_C, \\ \mathcal O_{\mathbb{P}^1}(1)\oplus\mathcal O_{\mathbb{P}^1}(-1), & x\in G_C, \qquad\quad \text{and}\ \ \chi(\widetilde{E}^0|_{\sigma^{-1}(x)})=2. \\ \mathbf{k}^2_{\sigma^{-1}(x)}, &x\in Q^0\setminus C^0, \end{cases} \end{equation} \tag{6.13} $$
Therefore, applying Proposition 1.13 from [12] to the projection $\sigma\colon X^0\to Q^0$ we obtain that
$$ \begin{equation} \overline{E}^0|_{Q^0\setminus G_C} \text{ is a locally free}\ {\mathcal O}_{Q^0\setminus G_C}\text{-sheaf} \end{equation} \tag{6.14} $$
and
$$ \begin{equation} \widetilde{E}^0|_{X^0\setminus \sigma^{-1}(G_C)}=\sigma^*(\overline{E}^0|_{Q^0 \setminus G_C}), \qquad \operatorname{Supp}(R^1\sigma_*\widetilde{E}^0)\subset G_C. \end{equation} \tag{6.15} $$
Since $x\in Q^0$, the fibre $p^{-1}(x)$ is isomorphic to $\mathbb{P}^1$, and
$$ \begin{equation} \mathcal L_0|_{p^{-1}(x)}\cong\mathcal O_{\mathbb{P}^1}(-s)^{\oplus r}, \end{equation} \tag{6.16} $$
so $p_*\mathcal{L}_0=0$. Taking into account the relative Serre duality for the locally trivial $\mathbb{P}^1$-bundle $p\colon Y^0\to Q^0$ we obtain that
$$ \begin{equation} \begin{gathered} \, \mathcal M_0=R^1p_*\mathcal L_0 \text{ is a locally free sheaf of rank} \\ \operatorname{rk}\mathcal M_0=h^1(\mathcal L_0|_{p^{-1}(x)})=-\chi(\mathcal L_0|_{p^{-1}(x)})=r(s-1), \qquad x\in Q^0. \end{gathered} \end{equation} \tag{6.17} $$
Restricting the exact triple (6.12) to the fibre $p^{-1}(x)$ we obtain the exact triple
$$ \begin{equation} 0\to\mathcal L_1|_{p^{-1}(x)}\to\mathcal O_{\mathbb{P}^1}(-s)^{\oplus r}\to\widetilde{E}^0|_{\sigma^{-1}(x)} \to0. \end{equation} \tag{6.18} $$
From (6.13) and (6.18) it follows that
$$ \begin{equation*} \chi(\mathcal L_1|_{p^{-1}(x)})=\chi(\mathcal L_0|_{p^{-1}(x)})-\chi(\widetilde{E}^0|_{\sigma^{-1} (x)})=-r(s-1)-2, \end{equation*} \notag $$
and by analogy with (6.17) we find that
$$ \begin{equation} \begin{gathered} \, \mathcal M_1=R^1p_*\mathcal L_1 \text{ is a locally free sheaf of rank} \\ \operatorname{rk}\mathcal M_1=h^1(\mathcal L_1|_{p^{-1}(x)})=-\chi(\mathcal L_1|_{p^{-1}(x)})=r(s-1)+2, \qquad x\in Q^0. \end{gathered} \end{equation} \tag{6.19} $$
As a consequence, applying the functor $R^ip_*$ to (6.12) we obtain the exact subsequence
$$ \begin{equation} 0\to\overline{E}^0\to\mathcal M_1\to\mathcal M_0\to\kappa\to0,\qquad \kappa:=R^1\sigma_*\widetilde{E}^0, \end{equation} \tag{6.20} $$
and in view of (6.15) we have the inclusion
$$ \begin{equation} \operatorname{Supp}(\kappa)\subset G_C. \end{equation} \tag{6.21} $$
Since $\mathcal{M}_0$ and $\mathcal{M}_1$ are locally free sheaves on a smooth three-dimensional variety $Q^0$, it follows (see, for example, Proposition 1.1 in [15]) that
$$ \begin{equation} \overline{E}^0 \text{ is a reflexive}\ \mathcal O_{Q^0}\text{-sheaf}. \end{equation} \tag{6.22} $$
On the other hand, again because of the local freeness of $\mathcal{M}_0$ and $\mathcal{M}_1$, applying the functor $\mathrm{Ext}_{\mathcal{O}_{Q^0}}^1(-,\mathcal{O}_{Q^0})$ to (6.20) and using the fact that $\kappa$ is a sheaf of dimension ${\leqslant0}$, we obtain an isomorphism of sheaves
$$ \begin{equation} \operatorname{Ext}_{\mathcal O_{Q^0}}^1(\overline{E}^0,\mathcal O_{Q^0})\cong\operatorname{Ext}_{\mathcal O_{Q^0}}^3( \kappa,\mathcal O_{Q^0}). \end{equation} \tag{6.23} $$
Since $\dim\kappa \leqslant 0$, the sheaf $\kappa$ is either zero or an Artinian sheaf, that is, $\kappa$ has a finite filtration with factors that are the residue fields $\mathbf{k}_x$ of points $x\in\operatorname{Supp}(\kappa)$. Since $\mathrm{Ext}_{\mathcal{O}_{Q^0}}^3 (\mathbf{k}_x,\mathcal{O}_{Q^0})\cong\mathbf{k}_x$, $x\in\operatorname{Supp}(\kappa)$, we obtain $\mathrm{Ext}_{\mathcal{O}_{Q^0}}^3(\kappa,\mathcal{O}_{Q^0}))=\chi(\kappa)$. From this and (6.23) it follows that
$$ \begin{equation} \operatorname{Supp}(\operatorname{Ext}_{\mathcal O_{Q^0}}^1(\overline{E}^0,\mathcal O_{Q^0}))=\operatorname{Supp}(\kappa) \end{equation} \tag{6.24} $$
and
$$ \begin{equation} d:=\chi(\operatorname{Ext}_{\mathcal O_{Q^0}}^1(\overline{E}^0,\mathcal O_{Q^0}))=\chi(\kappa). \end{equation} \tag{6.25} $$

Since $\sigma\colon\widetilde{X}\setminus S\xrightarrow{\cong}Q\setminus C$ is an isomorphism and $\operatorname{Sing}\widetilde{E} \subset\widetilde{X}\setminus S$, setting

$$ \begin{equation*} \overline{E}:=\sigma_*\widetilde{E}, \end{equation*} \notag $$
we have
$$ \begin{equation*} \sigma(\operatorname{Sing}\widetilde{E})=\operatorname{Sing}(\overline{E}|_{Q\setminus C}) \subset\operatorname{Sing}\overline{E}. \end{equation*} \notag $$
We also obtain an isomorphism of Artinian sheaves
$$ \begin{equation} \sigma_*\colon \operatorname{Ext}_{\mathcal O_{\widetilde{X}}}^1(\widetilde{E},\mathcal O_{\widetilde{X}}) \xrightarrow{\cong}\operatorname{Ext}_{\mathcal O_{Q}}^1(\overline{E},\mathcal O_{Q})|_{Q\setminus C}, \end{equation} \tag{6.26} $$
where
$$ \begin{equation} \operatorname{Supp}(\operatorname{Ext}_{\mathcal O_{Q}}^1(\overline{E},\mathcal O_{Q})|_{Q\setminus C})= \operatorname{Sing}(\overline{E}|_{Q\setminus C}). \end{equation} \tag{6.27} $$
Moreover, setting $c_3:=c_3(E)$ we have
$$ \begin{equation} c_3=c_3(\widetilde{E})=\chi(\operatorname{Ext}_{\mathcal O_{\widetilde{X}}}^1(\widetilde{E},\mathcal O_{ \widetilde{X}}))=\chi(\operatorname{Ext}_{\mathcal O_{Q}}^1(\overline{E},\mathcal O_{Q})|_{Q\setminus C}). \end{equation} \tag{6.28} $$
Here the first equality in (6.28) follows from (6.5), the second equality was proved in [15], Proposition 2.6, and the third equality follows from (6.26).

Put

$$ \begin{equation} Q_{\xi}:=Q\setminus D_{\xi}=Q^0\cup\operatorname{Sing}\overline{E}, \quad X_{\xi}:=\sigma^{-1}(Q_{\xi}), \quad \widetilde{E}_{\xi}:=\widetilde{E}|_{X_{\xi}}\quad\text{and} \quad \overline{E}_{\xi}:=\sigma_*\widetilde{E}_{\xi}. \end{equation} \tag{6.29} $$
By construction $\overline{E}^0=\overline{E}_{\xi}|_{Q^0}$, and therefore from (6.8), (6.14), (6.22) and (6.29) it follows that $\overline{E}_{\xi}$ is a reflexive sheaf which is locally free at points on the curve $C_{\xi}= C\setminus D_{\xi}$, in particular, at points of the divisor $D_{\xi'}$ on $C$:
$$ \begin{equation} \overline{E}_{\xi}\text{ is a reflexive sheaf, locally free at the points of}\ D_{\xi'}. \end{equation} \tag{6.30} $$
Taking now $\xi'$ instead of $\xi$ and assuming by analogy with (6.29) that
$$ \begin{equation} Q_{\xi'}:=Q\setminus D_{\xi'}, \quad X_{\xi'}:=\sigma^{-1}(Q_{\xi'}), \quad \widetilde{E}_{\xi'}:=\widetilde{E}|_{X_{\xi'}}\quad\text{and} \quad \overline{E}_{\xi'}:=\sigma_*\widetilde{E}_{\xi'}, \end{equation} \tag{6.31} $$
similarly to (6.30) we obtain:
$$ \begin{equation} \overline{E}_{\xi'} \text{ is a reflexive sheaf which is locally free at points of}\ D_{\xi}. \end{equation} \tag{6.32} $$
Consider the sheaf
$$ \begin{equation*} \overline{E}=\sigma_*\widetilde{E}. \end{equation*} \notag $$
From (6.7) we have
$$ \begin{equation} H^0(\overline{E})=0. \end{equation} \tag{6.33} $$
Note that by (6.29) and (6.31) $E_{\xi}=\overline{E}|_{Q_{\xi}}$ and $E_{\xi'}=\overline{E}|_{Q_{\xi'}}$. From this and (6.30) and (6.32) we find that $\overline{E}$ is a reflexive sheaf. Moreover, for any line $l'$ on $X$ that does not intersect another line $l$ we have an isomorphism $\beta:=(\sigma\circ\delta^{-1}|_l)\colon l'\xrightarrow{\sim}m:=\beta(l')$ such that $E|_{l'}\cong\overline{E}|_m$, hence $c_1(\overline{E}) =c_1(\overline{E}|_{m})=c_1(E|_{l'})=0$. Thus, $E$ is stable by virtue of (6.33). So
$$ \begin{equation} \overline{E}\text{ is a stable reflexive sheaf with}\ c_1(\overline{E})=0. \end{equation} \tag{6.34} $$
Note also that from (6.21), (6.29), (6.31) and the fact that $\overline{E}^0=\overline{E}|_{Q^0}$ it follows that $\kappa=R^1\sigma_*\widetilde{E}^0=R^1\sigma_*\widetilde{E}$, hence
$$ \begin{equation} \operatorname{Ext}_{\mathcal O_{Q}}^1(\overline{E},\mathcal O_{Q})|_{Q_0}=\operatorname{Ext}_{\mathcal O_{Q^0}}^1(\overline{E}^0, \mathcal O_{Q^0}) \end{equation} \tag{6.35} $$
and by (6.24) $\operatorname{Supp}(\mathrm{Ext}_{\mathcal{O}_{Q^0}}^1(\overline{E}^0, \mathcal{O}_{Q^0}))\subset G_C$. Since $G_C\cap(Q\setminus C)=\varnothing$, it follows from (6.35) that
$$ \begin{equation} \operatorname{Ext}_{\mathcal O_{Q}}^1(\overline{E},\mathcal O_{Q})=\operatorname{Ext}_{\mathcal O_{Q^0}}^1(\overline{E}^0,\mathcal O_{ Q^0})\oplus\operatorname{Ext}_{\mathcal O_{Q}}^1(\overline{E},\mathcal O_{Q})|_{Q\setminus C}. \end{equation} \tag{6.36} $$
Note that by Proposition 2.6 in [15] and (6.34) we have
$$ \begin{equation*} \overline{c}_3:=c_3(\overline{E})=\chi(\operatorname{Ext}_{\mathcal O_{Q}}^1(\overline{E},\mathcal O_{Q})), \end{equation*} \notag $$
and therefore (6.25), (6.28), (6.36) and the equality $\kappa=R^1\sigma_*\widetilde{E}$ give
$$ \begin{equation} \overline{c}_3=d+c_3, \qquad d=\chi(R^1\sigma_*\widetilde{E}). \end{equation} \tag{6.37} $$
Since, obviously, $R^i\sigma_*\widetilde{E}=0$, $i\geqslant2$, the Leray spectral sequence for the morphism $\sigma\colon \widetilde{X}\to Q$ gives $\chi(\widetilde{E})= \chi(\overline{E})-\chi(R^1\sigma_*\widetilde{E})$. Hence, taking (6.37) and (6.6) into account we have
$$ \begin{equation} \chi(\overline{E}))=\chi(E)+\chi(R^1\sigma_*\widetilde{E})=\chi(E)+d. \end{equation} \tag{6.38} $$
Set
$$ \begin{equation} c_2:=c_2(E) \quad\text{and}\quad \overline{c}_2:=c_2(\overline{E}). \end{equation} \tag{6.39} $$

Recall the general Riemann–Roch formula for an arbitrary coherent sheaf $\mathcal{E}$ of rank $r$ with Chern classes $\widetilde{c}_1$, $\widetilde{c}_2$ and $\widetilde{c}_3$ on a smooth projective three-dimensional Fano variety $X$ with canonical class $\omega_X$:

$$ \begin{equation} \chi(\mathcal E)=r\chi(\mathcal O_X)+\frac 16\widetilde{c}_1^3+\frac 12(\widetilde{c}_3-\widetilde{c}_1\widetilde{ c}_2)-\frac 14\omega_X(\widetilde{c}_1^2-2\widetilde{c}_2)+\frac{\widetilde{c}_1}{12}(c_2(\Omega _X)+\omega_X^2). \end{equation} \tag{6.40} $$
Applying this formula to the sheaf $E$ on $X=X_5$ (see (6.3)) and the sheaf $\overline{E}$ on $X=Q$, we obtain
$$ \begin{equation} \chi(E)=2+\frac{1}{2}c_3-c_2 \end{equation} \tag{6.41} $$
and
$$ \begin{equation} \chi(\overline{E})=2+\frac{1}{2}\overline{c}_3-\frac{3}{2}\overline{c}_2, \end{equation} \tag{6.42} $$
respectively. In addition, according to Theorem 3.1.(4), we have
$$ \begin{equation} \overline{c}_3\leqslant \begin{cases} \dfrac 12\overline{c}_2^2 &\text{if}\ c_2\ \text{is even}, \\ \dfrac 12(\overline{c}_2^2+1) & \text{if}\ c_2\ \text{is odd}. \end{cases} \end{equation} \tag{6.43} $$
From (6.38), (6.41) and (6.42) we find that $\overline{c}_2= \frac 23c_2-\frac 13d$. Substituting this relation and the equality $c_3=\overline{c}_3-d$, following from (6.37), into (6.43) we obtain the inequality
$$ \begin{equation} c_3=\overline{c}_3-d\leqslant \begin{cases} \dfrac 12\biggl(\dfrac 23c_2-\dfrac 13d\biggr)^2-d &\text{if}\ c_2\ \text{is even}, \\ \dfrac 12\biggl(\biggl(\dfrac 23c_2-\dfrac 13d\biggr)^2+1\biggr)-d &\text{if}\ c_2\ \text{is odd}. \end{cases} \end{equation} \tag{6.44} $$
Since $\overline{c}_2=\frac 23c_2-\frac 13d\geqslant0$ and $d\geqslant0$, assuming that $d=0$in (6.44) we obtain the main result of this subsection, which is the following theorem.

Theorem 6.1. Let $E$ be a rank $2$ stable reflexive sheaf of general type with ${c_1(\mkern-1mu E)\!=\!0}$, $c_2=c_2(E)>0$ and $c_3=c_3(E)$ on the variety $X=X_5$. Then the following inequalities are valid:

$$ \begin{equation} c_3\leqslant \begin{cases} \dfrac 29c_2^2 &\textit{if}\ c_2\ \textit{is even}, \\ \dfrac 29c_2^2+\dfrac 12 & \textit{if}\ c_2\ \textit{is odd}. \end{cases} \end{equation} \tag{6.45} $$

6.2. Rank $2$ stable reflexive sheaves of general type on $X_4$

Throughout this subsection $X=X_4$.

We list some known facts about the variety $X$ (see [1], Theorem 4.2, (iii), for instance).

(1) The base $B=B(X)$ of the family of lines on $X$ is isomorphic to the Jacobian $J(C)$ of a smooth curve $C$ of genus $2$.

(2) For an arbitrary line $l\in B$ the set $B_l=\{l'\in B\mid l'\cap l\ne\varnothing\}$ is a curve in $B$ isomorphic to the curve $C$.

(3) For an arbitrary line $l\in B$ either $N_{l/X}\cong\mathcal{O}_{\mathbb{P}^1}^{\oplus2}$ (the general case), or $N_{l/X}\cong\mathcal{O}_{\mathbb{P}^1}(1)\oplus\mathcal{O}_{\mathbb{P}^1}(-1)$.

(4) Let $\varphi\colon X\dashrightarrow \mathbb{P}^4$ be the linear projection of the variety $X\subset\mathbb{P}^5$ onto the space $\mathbb{P}^3$ from a line $l\subset X$ for which $N_{l/X}\cong\mathcal{O}_{\mathbb{P}^1}^{\oplus2}$. Then $\varphi(X)=\mathbb{P}^3$ and the morphism $\varphi\colon X\dashrightarrow\mathbb{P}^3$ is birational and decomposes into the Hironaka roof

$(6.46)$
Here $\delta^{-1}\colon X\dashrightarrow\widetilde{X}$ is the blow-up of $X$ centred at $l$, $S=\delta^{-1}(l)\simeq \mathbb{P}^1 \times\mathbb{P}^1$, $\sigma|_S\colon S\xrightarrow{\simeq}\sigma(S) =Q_2$ is an isomorphism of $S$ onto a smooth two-dimensional quadric $Q_2\subset\mathbb{P}^3$, and $\sigma^{-1} \colon Q\dashrightarrow\widetilde{X}$ is the blow-up of $Q$ centred in a smooth curve $C$ of type (2,3) on $Q_2$, so that $\widetilde{X}\simeq\mathbb{P}(\mathcal{I}_{C,\mathbb{P}^3})$, which is isomorphic to the curve $C$ from statement (1) above. In particular, the triple $0\to N_{C,Q_2}\to N_{C/\mathbb{P}^3}\to N_{Q_2/\mathbb{P}^3}|_{C}\to0$ is exact, where in view of the isomorphism $Q_2\simeq\mathbb{P}^1\times\mathbb{P}^1$ we obtain $N_{C/Q_2}\cong \mathcal{O}_C(D_{\mathrm{I}})$, $\deg D_{\mathrm{I}}=12$, and $N_{Q_2/\mathbb{P}^3}|_{C}\cong\mathcal{O}_{\mathbb{P}^3}(2H)|_{C} \cong\mathcal{O}_C(D_{\mathrm{II}})$, $\deg D_{\mathrm{II}}=10$, hence $\det N_{C/\mathbb{P}^3}\cong\mathcal{O}_C(D_{\mathrm{I}}+D_{\mathrm{II}})$. Note that the divisor $D_{\mathrm{I}}+D_{\mathrm{II}}$ on $C$ is very ample since $C$ is a curve of genus $2$.

(5) Let $\Gamma=\{(x,l)\in X\times B\mid x\in l\}$ be an incidence graph with projections $X\xleftarrow{p_1}\Gamma\xrightarrow{p_2}B$. It is known (see [1], Proposition 5.2) that the morphism $p_1\colon\Gamma\to X$ is a finite at a general point and $\dim\{x\in X\mid \dim p_1^{-1}(x)\geqslant1\}\leqslant0$.

Consider an extension

$$ \begin{equation} \xi\colon 0\to\mathcal O_{\mathbb{P}^3}(nH)\to F\xrightarrow{\varepsilon}\mathcal I_{C,\mathbb{P}^3}\to0. \end{equation} \tag{6.47} $$
We have $\mathcal{H}om(\mathcal{I}_{C,\mathbb{P}^3},\mathcal{O}_{\mathbb{P}^3}(nH)) \cong\mathcal{O}_{\mathbb{P}^3}(nH)$ on $\mathbb{P}^3$ and
$$ \begin{equation*} \begin{aligned} \, \mathcal M &=\operatorname{Ext}^1(\mathcal I_{C,\mathbb{P}^3},\mathcal O_{\mathbb{P}^3}(nH))\cong\operatorname{Ext} ^2(\mathcal O_C,\mathcal O_{\mathbb{P}^3}(nH)) \\ &\cong\det N_{C/\mathbb{P}^3} \otimes\mathcal O_{\mathbb{P}^3}(nH)\cong\mathcal O_C(D_{\mathrm{I}}+D_{\mathrm{II}}+nH) \end{aligned} \end{equation*} \notag $$
on $C$, where $n\gg0$, are very ample. Therefore, we can assume the following.

(1) $H^i(\mathcal{H}om(\mathcal{I}_{C,\mathbb{P}^3}, \mathcal{O}_{\mathbb{P}^3}(nH)))=0$, $i=1,2$, which means that the long exact sequence of $\operatorname{Ext}$-groups for the pair of sheaves $\mathcal{I}_{C,\mathbb{P}^3},\mathcal{O}_{\mathbb{P}^3}(nH)$ produces the isomorphisms

$$ \begin{equation*} \begin{aligned} \, \operatorname{Ext}^1(\mathcal Hom(\mathcal I_{C,\mathbb{P}^3},\mathcal O_{\mathbb{P}^3}(nH)) &\cong H^0(\operatorname{Ext}^1(\mathcal I_{C,\mathbb{P}^3}, \mathcal O_{\mathbb{P}^3}(nH)) \\ &\cong H^0(\operatorname{Ext}^2(\mathcal O_C,\mathcal O_{\mathbb{P}^3}(nH))=H^0(\mathcal M). \end{aligned} \end{equation*} \notag $$

(2) The triple (6.47) as an extension $\xi\in\operatorname{Ext}^1(\mathcal{H}om(\mathcal{I}_{C,\mathbb{P}^3}, \mathcal{O}_Q(nH))=H^0(\mathcal{M})$ is a section of the sheaf $\mathcal{M}$, whose the scheme of zeros $(\xi)_0$ is reduced, that is, it is a simple divisor $D_{\xi}=x_1+\dots+x_{5n+22}$. This means by Serre’s construction (see [15], § 4) that the sheaf $F$ in (6.47) is a reflexive sheaf of rank $2$ with $\operatorname{Sing} F=D_{\xi}$ and has the simplest singularities in the points $x_i$, that is, $\mathrm{Ext}^1(F,\mathcal{O}_Q)\cong\oplus_{i=1}^{3n+6}\mathbf{k}_{x_i}$. Therefore, according to Corollary 2.8 in [4], the projective spectrum $Y=\mathbb{P}(F)$ of the sheaf $F$ is a smooth variety. The epimorphism $\varepsilon$ in (6.47) corresponds to an embedding $i$ of the divisor $\widetilde{X}=\mathbb{P}(\mathcal{I}_{C,\mathbb{P}^3})$ into $Y$, and let $p\colon Y\to\mathbb{P}^3$ be the natural projection, so that $\sigma=p\circ i$. Note also that the sheaf

$$ \begin{equation*} L=\mathcal O_{Y/\mathbb{P}^3}(1)\otimes p^*\mathcal O(mH) \end{equation*} \notag $$
is ample for $m\gg0$, where $\mathcal{O}_{Y/\mathbb{P}^3}(1)$ is the Grothendieck sheaf on $Y=\mathbb{P}(F)$.

Let

$$ \begin{equation*} \begin{aligned} \, & E\text{ be a rank $2$ stable reflexive sheaf of general type} \\ &\text{with}\ c_1(E)=0\ \text{on}\ X_4; \text{ in particular,}\ H^0(E)=0. \end{aligned} \end{equation*} \notag $$

Repeating verbatim for the sheaf $E$ all arguments in (6.4)(6.44) with $Q$ and $Q^0$ replaced by $\mathbb{P}^3$ and $(\mathbb{P}^3)^0$, respectively, and using Hartshorne’s result for stable reflexive sheaves on $\mathbb{P}^3$ (see [15], Theorem 8.2, (b)), we obtain the following analogue of Theorem 6.1 for $X=X_4$.

Theorem 6.2. Let $E$ be a rank $2$ stable reflexive sheaf of general type with ${c_1(\mkern-1mu E)\!=\!0}$, $c_2=c_2(E)>0$ and $c_3=c_3(E)$ on the variety $X=X_4$. Then the following inequality is true:

$$ \begin{equation} c_3\leqslant c_2^2-c_2+2. \end{equation} \tag{6.48} $$


Bibliography

1. V. A. Iskovskih, “Fano 3-folds. I”, Math. USSR-Izv., 11:3 (1977), 485–527  mathnet  crossref  mathscinet  zmath
2. C. Almeida, M. Jardim and A. S. Tikhomirov, “Irreducible components of the moduli space of rank 2 sheaves of odd determinant on projective space”, Adv. Math., 402 (2022), 108363, 64 pp.  crossref  mathscinet  zmath
3. V. Antonelli, G. Casnati and O. Genc, “Even and odd instanton bundles on Fano threefolds”, Asian J. Math., 26:1 (2022), 81–118  crossref  mathscinet  zmath
4. E. Ballico and J. A. Wiśniewski, “On Bănică sheaves and Fano manifolds”, Compos. Math., 102:3 (1996), 313–335  mathscinet  zmath
5. C. Bănică, M. Putinar and G. Schumacher, “Variation der globalen Ext in Deformationen kompakter komplexer Räume”, Math. Ann., 250:2 (1980), 135–155  crossref  mathscinet  zmath
6. W. Barth and K. Hulek, “Monads and moduli of vector bundles”, Manuscripta Math., 25:4 (1978), 323–347  crossref  mathscinet  zmath
7. A. Bayer, E. Macri and Y. Toda, “Bridgeland stability conditions on threefolds I: Bogomolov–Gieseker type inequalities”, J. Algebraic Geom., 23:1 (2014), 117–163  crossref  mathscinet  zmath
8. T. Bridgeland, “Stability conditions on $K3$ surfaces”, Duke Math. J., 141:2 (2008), 241–291  crossref  mathscinet  zmath
9. G. Comaschi, M. Jardim, C. Martinez and Dapeng Mu, “Instanton sheaves: the next frontier”, São Paulo J. Math. Sci., 2023, 1–37, Publ. online  crossref
10. L. Costa and R. M. Miró-Roig, “Monads and instanton bundles on smooth hyperquadrics”, Math. Nachr., 282:2 (2009), 169–179  crossref  mathscinet  zmath
11. L. Ein, “Generalized null correlation bundles”, Nagoya Math. J., 111 (1988), 13–24  crossref  mathscinet  zmath
12. G. Ellingsrud and S. A. Strømme, “Stable rank-2 vector bundles on $\mathbb{P}^3$ with $c_1=0$ and $c_2=3$”, Math. Ann., 255:1 (1981), 123–135  crossref  mathscinet  zmath
13. D. Faenzi, “Even and odd instanton bundles on Fano threefolds of Picard number one”, Manuscripta Math., 144:1–2 (2014), 199–239  crossref  mathscinet  zmath
14. R. Hartshorne, “Stable vector bundles of rank 2 on $\mathbf{P}^3$”, Math. Ann., 238:3 (1978), 229–280  crossref  mathscinet  zmath
15. R. Hartshorne, “Stable reflexive sheaves”, Math. Ann., 254:2 (1980), 121–176  crossref  mathscinet  zmath
16. V. A. Iskovskikh and Yu. G. Prokhorov, “Fano varieties”, Algebraic geometry V, Encyclopaedia Math. Sci., 47, Springer, Berlin, 1999, 1–247  mathscinet  zmath
17. M. Jardim, D. Markushevich and A. S. Tikhomirov, “Two infinite series of moduli spaces of rank 2 sheaves on $\mathbb P^3$”, Ann. Mat. Pura Appl. (4), 196:4 (2017), 1573–1608  crossref  mathscinet  zmath
18. A. Kuznetsov, “Instanton bundles on Fano threefolds”, Cent. Eur. J. Math., 10:4 (2012), 1198–1231  mathnet  crossref  mathscinet  zmath
19. A. A. Kytmanov, A. S. Tikhomirov and S. A. Tikhomirov, “Series of rational moduli components of stable rank two vector bundles on $\mathbb P^3$”, Selecta Math. (N.S.), 25:2 (2019), 29, 47 pp.  crossref  mathscinet  zmath
20. H. Lange, “Universal families of extensions”, J. Algebra, 83:1 (1983), 101–112  crossref  mathscinet  zmath
21. E. Macrì and B. Schmidt, “Derived categories and the genus of space curves”, Algebr. Geom., 7:2 (2020), 153–191  crossref  mathscinet  zmath
22. R. M. Miró-Roig and G. Trautmann, “The moduli scheme $M(0,2,4)$ over $\mathbb P^3$”, Math. Z., 216:2 (1994), 283–315  crossref  mathscinet  zmath
23. P. Rao, “A note on cohomology modules of rank two bundles”, J. Algebra, 86:1 (1984), 23–34  crossref  mathscinet  zmath
24. B. Schmidt, “A generalized Bogomolov–Gieseker inequality for the smooth quadric threefold”, Bull. Lond. Math. Soc., 46:5 (2014), 915–923  crossref  mathscinet  zmath
25. B. Schmidt, “Rank two sheaves with maximal third Chern character in three-dimensional projective space”, Mat. Contemp., 47 (2020), 228–270  crossref  mathscinet  zmath
26. B. Schmidt, “Bridgeland stability on threefolds: some wall crossings”, J. Algebraic Geom., 29:2 (2020), 247–283  crossref  mathscinet  zmath
27. B. Schmidt, “Sheaves of low rank in three-dimensional projective space”, Eur. J. Math., 9:4 (2023), 103, 71 pp.  crossref  mathscinet  zmath
28. I. R. Shafarevich, Basic algebraic geometry, v. 1, Varieties in projective space, 3rd ed., Springer, Heidelberg, 2013, xviii+310 pp.  crossref  mathscinet  zmath
29. V. K. Vedernikov, “Moduli of stable vector bundles of rank 2 on $P_3$ with fixed spectrum”, Math. USSR-Izv., 25:2 (1985), 301–313  mathnet  crossref  mathscinet  zmath  adsnasa
30. V. K. Vedernikov, “The moduli of super-null-correlation bundles on $\mathbf P_3$”, Math. Ann., 276:3 (1987), 365–383  crossref  mathscinet  zmath

Citation: D. A. Vasil'ev, A. S. Tikhomirov, “Moduli of rank $2$ semistable sheaves on rational Fano threefolds of the main series”, Sb. Math., 215:10 (2024), 1269–1320
Citation in format AMSBIB
\Bibitem{VasTik24}
\by D.~A.~Vasil'ev, A.~S.~Tikhomirov
\paper Moduli of rank~$2$ semistable sheaves on~rational Fano threefolds of the main series
\jour Sb. Math.
\yr 2024
\vol 215
\issue 10
\pages 1269--1320
\mathnet{http://mi.mathnet.ru/eng/sm10087}
\crossref{https://doi.org/10.4213/sm10087e}
\mathscinet{https://mathscinet.ams.org/mathscinet-getitem?mr=4849358}
\adsnasa{https://adsabs.harvard.edu/cgi-bin/bib_query?2024SbMat.215.1269V}
\isi{https://gateway.webofknowledge.com/gateway/Gateway.cgi?GWVersion=2&SrcApp=Publons&SrcAuth=Publons_CEL&DestLinkType=FullRecord&DestApp=WOS_CPL&KeyUT=001406213400001}
\scopus{https://www.scopus.com/record/display.url?origin=inward&eid=2-s2.0-85216091872}
Linking options:
  • https://www.mathnet.ru/eng/sm10087
  • https://doi.org/10.4213/sm10087e
  • https://www.mathnet.ru/eng/sm/v215/i10/p3
    Erratum
    This publication is cited in the following 2 articles:
    Citing articles in Google Scholar: Russian citations, English citations
    Related articles in Google Scholar: Russian articles, English articles
    Математический сборник Sbornik: Mathematics
     
      Contact us:
     Terms of Use  Registration to the website  Logotypes © Steklov Mathematical Institute RAS, 2025