No, bauxite processing is pretty benign. Strong hydroxides are used to extract the aluminum. Conveniently, a fraction of the gallium in the ore also leaches into the same hydroxide solution, so you can extract it by processing the waste solution after the aluminum has been precipitated. Any ore that uses strong alkali extraction is a candidate for cheap gallium extraction.

Bauxite processing produces iron, silicate, and similar minerals. Nothing anyone would identify as toxic. The alkali hydroxide solutions are recycled because they are one of the most expensive inputs. Nothing to worry about as such things go.