Well yes in a way, the extension settings include the prompt they're using, so you can formulate your own prompt along the same lines.
I think the extension is only useful for someone who spends a lot of time doing research on arxiv.org, (and if the quality of the summary is good enough, the jury's still out on that one)