jaxlib._jax.XlaRuntimeError #32575
-
|
Greetings, I am facing this issue in using Python Biogeme. Could you please sort out this. Thanks |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 7 replies
-
|
Hello – from the error message it looks like you're running a script that attempts to allocate more memory than is available on your device. To fix this you'll need to reduce the size of the arrays in your program. If you'd like more specific help with this, please try putting together a minimal reproducible example. |
Beta Was this translation helpful? Give feedback.
-
|
This is an example. When I make it more complex by including additional explanatory variables, it produces the error shown above. This issue occurs not only with this script but also with other models, which were all working successfully before the Biogeme update based on JAX. I would greatly appreciate your help. Thank you! notebook json{ |
Beta Was this translation helpful? Give feedback.
-
|
Instead of pasting the raw json source of a jupyter notebook (which is difficult to read and understand), could you put together a simple code snippet similar to what's described at the link I posted above? |
Beta Was this translation helpful? Give feedback.
-
# ----------------------
# Hybrid Choice Model: Walk vs Others
# ----------------------
import os
import numpy as np
import pandas as pd
from biogeme.biogeme import BIOGEME
import biogeme.database as db
from biogeme.expressions import Beta, Variable, Draws, LinearTermTuple, LinearUtility, Numeric, MonteCarlo, log, exp, Elem, MultipleProduct, NormalCdf
from biogeme.models import logit
from biogeme.results_processing import get_pandas_estimated_parameters
# ----------------------
# User settings
# ----------------------
DATA_FILE = "C:\\Users\\tariq\\Desktop\\MotionTag Final Data App\\MT_walk.csv"
MODEL_NAME = "hybrid_walk_vs_others"
NUMBER_OF_DRAWS = 1000
CHOICE_VAR = "CHOICE"
INDICATORS = ["ACCW1","ACCW2","ACCW3","ACCW4"]
DISCRETE_VALUES = [1,2,3,4,5]
MISSING_VALUES = [-1,-2,99]
# ----------------------
# Read CSV and prepare database
# ----------------------
df = pd.read_csv(DATA_FILE)
df = df[df[CHOICE_VAR].isin([1,2])]
if "normalized_weight" not in df.columns:
df["normalized_weight"] = 1.0
database = db.Database("data", df)
# ----------------------
# Variables
# ----------------------
Choice = Variable(CHOICE_VAR)
Den_Recretional_Act_Origin = Variable("Den_Recretional_Act_Origin")
# ----------------------
# Latent variable accw
# ----------------------
b_Den = Beta("struct_accw_Den", 0.0, None, None, 0)
sigma_accw = Beta("struct_accw_sigma", 1.0, None, None, 0)
accw_linear = LinearUtility([LinearTermTuple(b_Den, Den_Recretional_Act_Origin)])
accw = accw_linear + sigma_accw * Draws("struct_accw_error", "NORMAL_MLHS_ANTI")
# ----------------------
# Ordered probit for measurement equations
# ----------------------
def ordered_probit(continuous_value, scale_parameter, values, thresholds):
probs = {}
probs[values[0]] = NormalCdf((thresholds[0]-continuous_value)/scale_parameter)
for i in range(1,len(values)-1):
probs[values[i]] = NormalCdf((thresholds[i]-continuous_value)/scale_parameter) - \
NormalCdf((thresholds[i-1]-continuous_value)/scale_parameter)
probs[values[-1]] = 1 - NormalCdf((thresholds[-1]-continuous_value)/scale_parameter)
return probs
# Example measurement equation
log_d1, log_d2 = Beta("log_delta_1", np.log(0.3), None, None,0), Beta("log_delta_2", np.log(0.8), None, None,0)
d1, d2 = exp(log_d1), exp(log_d2)
thresholds = [-d1-d2, -d1, d1, d1+d2]
def measurement_likelihood(latent, indicators):
factors = []
for ind in indicators:
intercept = Beta(f"meas_intercept_{ind}", 0.0, None, None, 0)
loading = Numeric(1.0) if ind == indicators[0] else Beta(f"meas_coeff_{ind}",0.0,None,None,0)
scale = Beta(f"meas_scale_{ind}",1.0,None,None,0)
probs = ordered_probit(intercept + loading*latent, scale, DISCRETE_VALUES, thresholds)
for mv in MISSING_VALUES:
probs[mv] = Numeric(1.0)
factors.append(Elem(probs, Variable(ind)))
return MultipleProduct(factors)
meas_like = measurement_likelihood(accw, INDICATORS)
# ----------------------
# Choice utilities
# ----------------------
ASC_others = Numeric(0.0)
ASC_walk = Beta("asc_walk",0.0,None,None,0)
beta_accw_walk = Beta("beta_accw_walk",0.0,None,None,0)
beta_Den_walk = Beta("beta_Den_walk",0.0,None,None,0)
v = {
1: ASC_others,
2: ASC_walk + beta_accw_walk*accw + beta_Den_walk*Den_Recretional_Act_Origin
}
# ----------------------
# Log-likelihood
# ----------------------
choice_like = logit(v, None, Choice)
conditional_like = choice_like * meas_like
loglike = log(MonteCarlo(conditional_like))
# ----------------------
# Estimate model
# ----------------------
biogeme = BIOGEME(database, loglike, number_of_draws=NUMBER_OF_DRAWS)
biogeme.model_name = MODEL_NAME
results = biogeme.estimate()
print("Number of estimated parameters:", results.number_of_parameters)
print("Final log-likelihood:", results.final_log_likelihood)
# ----------------------
# Show results
# ----------------------
df_res = get_pandas_estimated_parameters(results)
print(df_res) |
Beta Was this translation helpful? Give feedback.
-
|
It looks like this code mainly depends on the But broadly speaking: the error you're seeing indicates that you're running a computation that uses more memory than is available on your device. The best approach will probably involve splitting your training into batches which can be run independently. I'd suggest looking into whether this kind of approach is supported by |
Beta Was this translation helpful? Give feedback.
-
|
Thanks alot |
Beta Was this translation helpful? Give feedback.
It looks like this code mainly depends on the
biogemepackage, which is likely doing a lot of things under the hood. I'm not familiar with this package, so I won't be able to comment on whether you're using it correctly. I think you'd probably find better results by asking about your out-of-memory issue in a forum specific to thebiogemepackage, if possible.But broadly speaking: the error you're seeing indicates that you're running a computation that uses more memory than is available on your device. The best approach will probably involve splitting your training into batches which can be run independently. I'd suggest looking into whether this kind of approach is supported by
biogeme.