If you are struggling to understand the README, I highly recommend the book Statistical Rethinking: A Bayesian Course with Examples in R and Stan by Richard McElreath. Although the examples are in R, the same concepts apply to Pyro (and NumPyro)
For those with more experience how does (Num)Pyro compare with PyMC? I haven’t had the good fortune of working with any of these libraries since before Pyro (and presumably numpyro), and with PyMC3 back when it used Theano under the hood.
Are the two libraries in competition? Or complimentary? I’ve been playing with PyMC for a personal project and am curious what I might gain from investigating (Num)Pyro?
I would say that, at least for me, PyMC’s main advantage was in DX. I just found model construction much more straightforward and better aligned with how I wanted to assemble the model.
I tired both a while back, but nothing too big or serious. One thing that numpyro benefits from is JAX's speed, so it might be faster for larger models. Though PyTensor, which is the backend for PyMC can apparently also generate JAX code, so the difference might not be drastic. The PyMC API also seemed to me easier to get started with for those learning Bayesian stats.
One thing I remember that I disliked about PyMC was the PyTensor API, it feels too much like Theano/TensorFlow. I much prefer using JAX for writing custom models.
This is maybe not the place, but we did some apples to apples comparisons between PyMC, Dynesty, and the Julia Turing.jl package.
A little to my surprise, despite being a Julia fan, Turing really outperformed both the Python solutions.
I think JAX should be competitive in raw speed, so it might come down to the maturity of the samplers we used.
You'll lose a lot of the PyMC convenience functions with Numpyro but gain a lot of control and flexibility over your model specification. If you're doing variational inference, Numpyro is the way to go.
You can use the Numpyro NUTS sampler in PyMC with pm.sample(nuts_sampler="numpyro") and it will significantly speed up sampling. It is less stable in my experience.
If you are struggling to understand the README, I highly recommend the book Statistical Rethinking: A Bayesian Course with Examples in R and Stan by Richard McElreath. Although the examples are in R, the same concepts apply to Pyro (and NumPyro)
[1] https://www.goodreads.com/book/show/26619686-statistical-ret...
I read through the second edition, and would recommend as well [1] (has numpyro, pyro, pymc code). He also has a great lecture series on youtube [2]
[1] https://xcelab.net/rm/
[2] https://www.youtube.com/watch?v=FdnMWdICdRs&list=PLDcUM9US4X...
There are examples from the book done in PyMC:
https://github.com/pymc-devs/pymc-resources/tree/main/Rethin...
or, for first edition
https://github.com/pymc-devs/pymc-resources/tree/main/Rethin...
For those with more experience how does (Num)Pyro compare with PyMC? I haven’t had the good fortune of working with any of these libraries since before Pyro (and presumably numpyro), and with PyMC3 back when it used Theano under the hood.
Are the two libraries in competition? Or complimentary? I’ve been playing with PyMC for a personal project and am curious what I might gain from investigating (Num)Pyro?
I would say that, at least for me, PyMC’s main advantage was in DX. I just found model construction much more straightforward and better aligned with how I wanted to assemble the model.
This textbook/walkthrough is great: https://a.co/d/9dWXDTK
Thanks for the textbook link, this looks right up my alley!
I tired both a while back, but nothing too big or serious. One thing that numpyro benefits from is JAX's speed, so it might be faster for larger models. Though PyTensor, which is the backend for PyMC can apparently also generate JAX code, so the difference might not be drastic. The PyMC API also seemed to me easier to get started with for those learning Bayesian stats.
One thing I remember that I disliked about PyMC was the PyTensor API, it feels too much like Theano/TensorFlow. I much prefer using JAX for writing custom models.
This is maybe not the place, but we did some apples to apples comparisons between PyMC, Dynesty, and the Julia Turing.jl package.
A little to my surprise, despite being a Julia fan, Turing really outperformed both the Python solutions. I think JAX should be competitive in raw speed, so it might come down to the maturity of the samplers we used.
You'll lose a lot of the PyMC convenience functions with Numpyro but gain a lot of control and flexibility over your model specification. If you're doing variational inference, Numpyro is the way to go.
You can use the Numpyro NUTS sampler in PyMC with pm.sample(nuts_sampler="numpyro") and it will significantly speed up sampling. It is less stable in my experience.
I find the PyMC situation confusing. But PyMC should be able to use JAX in the backend: https://www.pymc.io/projects/examples/en/latest/samplers/fas...
What is this used for?
MCMC and Bayesian models.
https://en.wikipedia.org/wiki/Probabilistic_programming
They're trying to drum up interest in Jax. They do this time to time, but yeah, good for them doing what they like.
side note: the Pyro used here - or actually pyro-ppl - is not to be confused with Pyro as of python-remote-objects..