For questions involving $\lfloor x \rfloor$, it is sometimes easier to fix this floor value, because as the integer part of $x$, one integer corresponds to infinitely many $x$.  Before using algebra to solve anything, try a few small numbers for $\lfloor x\rfloor$.

This problem requires that $x\ge 0$, so it means $\lfloor x\rfloor \geq 0$.  What if $\lfloor x\rfloor = 0$?  The equation becomes $$\lfloor x^2 \rfloor = 0.$$  This means $0\leq x^2 < 1$, and thus $0\leq x < 1$.

Now what if $\lfloor x \rfloor = 1$?  The equation becomes $$\lfloor x^2 \rfloor = 1,$$ and so $1 \leq x^2 < 2$, which means $1\leq x < \sqrt{2}$.

So, in general, we can let $\lfloor x\rfloor = n$, and solve it in terms of $n$.  The full solution includes all these separate intervals.