In the Generalized Beta distribution of the second kind (GB2), where a, p, and q are shape parameters and b is a scale parameter, the pdf is defined on $\mathbb{R}_+$ by: $$ GB2(y;a,b,p,q) = \frac{|a|y^{ap-1}}{b^{ap}B(p,q)(1+(y/b)^a)^{p+q}} $$ In the Pareto distribution, the tail index is just the single shape parameter (usually denoted by α).
In the more complicated GB2, if the four parameters are known, is there a closed-form solution for the tail index? Do all four parameters occur in it?
The "tail index" of a distribution function $F$ describes the rate of decrease of the survival function $1-F(y)$. (Thus, since the $b$ is merely a scale parameter, it cannot possibly influence the tail index.)
The question gives the derivative $f(y)=F^\prime(y)$. We can determine the asymptotic rate of decrease of $f$ by inspection, since for very large $y$, $1$ is much smaller than $y/b$ and the rate doesn't depend on any multiplicative constants. Thus
$$f(y) = \frac{|a|y^{ap-1}}{b^{ap}B(p,q)(1+(y/b)^a)^{p+q}} \approx C \frac{y^{ap-1}}{(y^a)^{(p+q)}} = Cy^{-aq-1}$$
where $C$ collects all the multiplicative constants. In general, whenever a function proportional to a power $k$ of $y$ is integrated, the result is proportional to $y^{k+1}$ (provided $k\ne -1$, but that cannot be the case for otherwise $F$ would diverge instead of approaching $1$ as a limit). Therefore $1-F$ must behave asymptotically like a multiple of $y^{-aq}$. Such functions are said to have a "tail index" of $1/(aq)$--that is, the negative reciprocal of the power.
The foregoing can be made rigorous by expanding $f$ about $\infty$ to first order. The expansion is provided by the Binomial Theorem or Taylor's Theorem (you express $f$ in terms of $x=1/y$, expand around $x=0$, and then rewrite the result in terms of $y$) and is absolutely convergent for $|y| \gt b$, justifying integrating the expansion term-by-term when going from $f$ to $1-F$. You need to estimate the error in this expansion and show it is a "slowly varying function," but doing so is straightforward, requiring no new ideas or techniques.
This is just an extended comment (and definitely not an answer) adding some details and checks to @whuber 's answer. (Mathematica is used when some code is needed.)
From Qi (2010) the tail index of the distribution function $F$ is $1/\gamma$ defined by
$$1-F(y)=y^{-1/\gamma} L(y)$$
for $y>0$ with the function $L$ satisfying
$$\lim_{t\rightarrow \infty} {{L(t y)}\over{L(t)}} = 1$$
We start with the density function for the generalized beta distribution of the second kind:
f[y_] := (Abs[a]/(b^(a p) Beta[p, q])) y^(a p - 1)/(1 + (y/b)^a)^(p + q)
As noted by @whuber the term $\left(\left(\frac{y}{b}\right)^a+1\right)^{-p-q}$ can be replaced by $\left(\left(\frac{y}{b}\right)^a\right)^{-p-q}$ when $y$ is large:
f4LargeY[y_] := FullSimplify[f[y] //. (1 + (y/b)^a)^(-p - q) -> y^(-a (p + q)) b^(a (p + q))]
which simplifies to
$$\frac{\left| a\right| b^{a q} y^{-a q-1}}{B(p,q)}$$
We should check on that assumption by taking the limit of the ratio of the two functions to see if that ratio approaches 1 as $y\rightarrow \infty$:
Limit[f[y]/f4LargeY[y], y -> \[Infinity], Assumptions -> {a > 0, b > 0, p > 0, q > 0}]
(* 1 *)
and the limit is 1.
We see that $1-F(y)$ is approximately $\int_y^{\infty } \text{f4LargeY}(t) \, dt$ for large enough values of $y$:
OneMinusF = Integrate[f4LargeY[t], {t, y, \[Infinity]},
Assumptions -> {a > 0, b > 0, p > 0, q > 0, y > 0}] /. (b/y)^(a q) -> y^(-a q) b^(a q)
We have $1-F(y) \approx \frac{b^{a q} y^{-a q}}{q B(p,q)}$ for large $y$. If we let $L(y)=\frac{b^{a q}}{q B(p,q)}$, then this function satisfies the requirement in Qi (2010) as $L(y)$ is just a constant.
L[y_] := b^(a q)/(q Beta[p, q])
Limit[L[t y]/L[y], t -> \[Infinity]]
(* 1 *)
So (and, yes, this is a bit of overkill in the use of Solve) we can solve for the tail index
FullSimplify[Solve[OneMinusF == y^(-tailIndex) L[y], tailIndex],
Assumptions -> {y > 0, a > 0, b > 0, p > 0, q > 0}][[1, 1]]
(* tailIndex -> a q *)
