New error message when building statespace_graph

[rklees]

I installed the latest version of pymc-experimental and get an error message pointing to

ssm_model.build_statespace_graph(data['meas'], mode="JAX")

The error message is

TypeError: Cannot convert Type Scalar(float64, shape=()) (of Variable sigma_SA_cycle) into Type Vector(float64, shape=(1,)). You can try to manually convert sigma_SA_cycle into a Vector(float64, shape=(1,)).

It refers to the pytensor variable

sigma_SA_cycle = pm.Gamma("sigma_SA_cycle", alpha=2, beta=5)

which is defined in pm.model()

I have installed pytensor version 2.18.6.

Any idea of how to fix this?

[jessegrabowski]

Looks like a bug related to the refactor a did a few months back. I'll have a look today. Thanks for flagging this.

[rklees]

I don’t exclude that other variables are also affected which are defined in pm.Model will also generate the same error message. In my case, the pm.Model file looks like: with pm.Model(coords=coords) as pymc_model: P0_diag = pm.Gamma("P0_diag", alpha=2, beta=5, dims=P0_dims[0]) P0 = pm.Deterministic("P0", pt.diag(P0_diag), dims=P0_dims) initial_trend = pm.Normal("initial_trend", dims=initial_trend_dims) sigma_trend = pm.Gamma("sigma_trend", alpha=2, beta=10, dims=sigma_trend_dims) annual_cycle = pm.Normal("annual_cycle", sigma=5, dims=annual_cycle_dims) sigma_annual_cycle = pm.Gamma("sigma_annual_cycle", alpha=2, beta=5) SA_cycle = pm.Normal("SA_cycle", sigma=5, dims=SA_cycle_dims) sigma_SA_cycle = pm.Gamma("sigma_SA_cycle", alpha=2, beta=5) sigma_obs = pm.Gamma("sigma_obs", alpha=2, beta=5, dims=('observed_state',)) From: Jesse Grabowski ***@***.***> Reply to: pymc-devs/pymc-experimental ***@***.***> Date: Wednesday, 27 March 2024 at 14:08 To: pymc-devs/pymc-experimental ***@***.***> Cc: Roland Klees ***@***.***>, Author ***@***.***> Subject: Re: [pymc-devs/pymc-experimental] New error message when building statespace_graph (Issue #318) You don't often get email from ***@***.*** Learn why this is important<https://aka.ms/LearnAboutSenderIdentification> Looks like a bug related to the refactor a did a few months back. I'll have a look today. Thanks for flagging this. — Reply to this email directly, view it on GitHub<#318 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AWKBIZJE7AOL3QQAVNNKNNTY2KZEPAVCNFSM6AAAAABFJMAWV6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMRSG4ZDKNJVGE>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[rklees]

Has this issue been solved in version .18 ??

[jessegrabowski]

No, but I'm working on patching it, plus updates to the example notebooks to reflect the changes to expected shapes and dims. I'll push a PR this weekend.

In the meantime you will have to give a singleton shape (via the shape = (1,)) to the parameters that are raising this error. The error is happening because I missed some parameters when I made the decision to require parameters that are never arrays to be scalars (instead of 1d arrays of a single element). In some cases, theparams_info was updated but the required shape of the parameter was not, so there's a mismatch. You have to go with what the error messages tells you to do for now.

[rklees]

Great. Thanks. I just opened an issue (# 325), though I must admit it is more a question. However, I don’t know where to put questions. Sorry for that. R Klees From: Jesse Grabowski ***@***.***> Reply to: pymc-devs/pymc-experimental ***@***.***> Date: Friday, 5 April 2024 at 14:59 To: pymc-devs/pymc-experimental ***@***.***> Cc: Roland Klees ***@***.***>, Author ***@***.***> Subject: Re: [pymc-devs/pymc-experimental] New error message when building statespace_graph (Issue #318) No, but I'm working on patching it, plus updates to the example notebooks to reflect the changes to expected shapes and dims. I'll push a PR this weekend. — Reply to this email directly, view it on GitHub<#318 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AWKBIZMCF6ZR27ZOIIYFS2LY32N2XAVCNFSM6AAAAABFJMAWV6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAMZZG42DKOBWGI>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[jessegrabowski]

Happy to talk on here, or on the pymc discourse. Whatever is most convenient for you.

[rklees]

I wonder if someone could send me a link to version 0.0.16 with the sigma_xxx are indeed the disturbance standard deviations and not the disturbance variances. In the original 0.0.16 version, sigma_xxx were in fact disturbance variances (the reason for asking is that both version 0.0.17 and version 0.0.18 do not work because of the bug mentioned in this post.

[jessegrabowski]

#326 will close this issue, and I'll merge it after the CI runs in about an hour. In the meantime, you can request specific versions by using pip, see here

[rklees]

The point is that version 0.0.16 still suffer from the fact that sigma_xxx is in fact sigma2_xxx, i.e., a variance. It is not clear to me whether the priors sigma_xxx are also variances or whether the distributions I need to define for sigma_xxx are distributions for variances etc. Without this knowledge, version 0.0.16 is not useful for unobserved components. The versions 0.0.17 and 0.0.18 generate error messages when building the graph, likely caused by inconsistencies about the shape of input quantities sigma_xxx. Unfortunately, I need a running version for an on-going Master project, so a quick fix would help a lot. Thanks. Roland K From: Jesse Grabowski ***@***.***> Reply to: pymc-devs/pymc-experimental ***@***.***> Date: Monday, 8 April 2024 at 21:59 To: pymc-devs/pymc-experimental ***@***.***> Cc: Roland Klees ***@***.***>, Author ***@***.***> Subject: Re: [pymc-devs/pymc-experimental] New error message when building statespace_graph (Issue #318) #326<#326> will close this issue, and I'll merge it after the CI runs in about an hour. In the meantime, you can request specific versions by using pip, see here <https://stackoverflow.com/questions/5226311/installing-specific-package-version-with-pip> — Reply to this email directly, view it on GitHub<#318 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AWKBIZLTYPDMJFDXLC6BBT3Y4LZKXAVCNFSM6AAAAABFJMAWV6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDANBTGU2DEMBXHE>. You are receiving this because you authored the thread.Message ID: ***@***.***>

[jessegrabowski]

You can try directly installing #326 if you can't wait for it to be merged. Be aware that it breaks old code, because all singleton parameters now do not expect parameters. For example, the cycle dampening parameter, as well as all sigmas. I updated and re-ran all examples in the structural notebook, so you can check that for guidance.

[rklees]

I installed from #326 and downgraded to pymc=5.12.0. After that, I could run the Structural Time Series modeling notebook you provided and got the same results. However, running pymc-experimental on a synthetic datasets, which I am using already for months, provides results, which are completely different from those obtained with pymc-experimental version 0.0.16. For instance, HDI intervals for the posterior predictive, are completely unrealistic, i.e., orders of magnitude too large.