Hi Edmond Zhang
The biggest dataset I use so far is 20k rows, and it has been running very smoothly with Altair Interactive Plot. A trick behind that is we should store the data to file, and reference the data by filename instead. This lets me plot the full dataset without having to embed it in the notebook.
However, the Altair is based on Vega and Vega-Lite, which generate web-based views using Canvas or SVG. In general Canvas is better than SVG when plotting a large dataset (SVG will be slow because of the number of DOM objects). So I would recommend to switch to Canvas.
The other better solution is WebGL, there is a WebGL render for Vega, but I doubt Altair lib have the support for that at this time.
I hope that helps.