In this work, we study the Biclique-Free Vertex Deletion problem: Given a graph $G$ and integers $k$ and $i \le j$, find a set of at most $k$ vertices that intersects every (not necessarily induced) biclique $K_{i, j}$ in $G$. This is a natural generalization of the Bounded-Degree Deletion problem, wherein one asks whether there is a set of at most $k$ vertices whose deletion results in a graph of a given maximum degree $r$. The two problems coincide when $i = 1$ and $j = r + 1$. We show that Biclique-Free Vertex Deletion is fixed-parameter tractable with respect to $k + d$ for the degeneracy $d$ by developing a $2^{O(d k^2)} \cdot n^{O(1)}$-time algorithm. We also show that it can be solved in $2^{O(f k)} \cdot n^{O(1)}$ time for the feedback vertex number $f$ when $i \ge 2$. In contrast, we find that it is W[1]-hard for the treedepth for any integer $i \ge 1$. Finally, we show that Biclique-Free Vertex Deletion has a polynomial kernel for every $i \ge 1$ when parameterized by the feedback edge number. Previously, for this parameter, its fixed-parameter tractability for $i = 1$ was known (Betzler et al., 2012) but the existence of polynomial kernel was open.