Keywords: Bayesian analysis; Data augmentation; Prior distributions; Probit models; Rate of convergence

We introduce a set of new Markov chain Monte Carlo algorithms for Bayesian analysis of the multinomial probit model. Our Bayesian representation of the model places a new, and possibly improper, prior distribution directly on the identifiable parameters and thus is relatively easy to interpret and use. Our algorithms, which are based on the method of marginal data augmentation, involve only draws from standard distributions and dominate other available Bayesian methods in that they are as quick to converge as the fastest methods but with a more attractive prior specification. Computer code for our algorithms is publicly available.

Keywords: Stochastic approximation, random effect, spatial model, MCMC

In recent decades, much effort has been devoted to maximum likelihood fitting of generalized linear mixed models with independent and dynamic random effects. In this paper, we consider a class of parametric random effect models which includes the generalized linear mixed models as a sub-class. We consider independent random effects, dynamic random effects and spatial random effects. A general two-stage Monte Carlo stochastic approximation based algorithm is developed to maximize the likelihood function of this class of models. The algorithm combines the features of the MCMC methods and some recent breakthroughs in stochastic approximation field to optimize the speed of convergence. A proper stopping criterion is also included so excuting the algorithm is automatic once the desired precision is given. Simulation studies and real examples have been explored and the proposed algorithms show satisfactory results.

Keywords: exponential distribution; extreme value distribution; multinomial logistic regression; discrete choice model; Markov chain Monte Carlo

This article describes an augmented variables method for sampling the posterior distribution of multinomial logit models, where the probability that unit $i$ assumes the value $m$ is proportional to $\exp(g_m(\bx_i|\beta))$. The method cycles between sampling a set of exponential random variables with rates $\exp(g_m(\bx_i|\beta))$, proposing new parameters from a distribution estimated using transformations of the exponential variables, and accepting or rejecting the proposal according to a Metropolis-Hastings probability. The method requires neither tuning nor iterative root finding to construct proposal distributions. It is fast, adaptive, and allows either categorical or continuous covariates. If the $g_m$ have distinct parameters then conditioning on the exponential variables renders the parameters of each $g_m$ independent in the complete data likelihood. Methods are discussed for choices of $g_m$ corresponding to multinomial logistic regression, ordinal logit models, hierarchical multinomial logistic regression, partial credit models, and generalized additive multinomial logit models.

Keywords: Bayesian inference, gene expression, Gibbs' sampler, M-A plots, microarray normalization.

One of the more challenging, yet easily overlooked, aspects of the analysis of microarrays is how to normalize arrays so that comparisons can be made across arrays. Most studies that utilize microarrays to detect differential gene expression between samples find the data only enable one to conclude that a handful of genes are differentially expressed. The basic idea here is to use the genes that are not differentially expressed to conduct the normalization. Of course, since one can't determine which genes are differentially expressed until the normalization is conducted, this is a non-trivial problem. Here a general framework and computational method (using the Gibbs' sampler) is devised to allow for such normalization.

Keywords:

Bayesian model averaging (BMA) for the linear regression problem with $p$ correlated predictors requires calculations on the order of $2^p$, prohibiting enumeration of the model space for moderate to large $p$. With orthogonality of the columns of the design matrix, the number of calculations can be reduced to order $p$. In most real applications, the variables of interest are not orthogonal. One can, however, augment the original design matrix by adding rows, so that the resulting "complete" design matrix has mutually orthogonal columns. The introduction of the complete design and latent variables for the augmented responses, combined with Rao-Blackwellization, leads to an order $p$ implementation for BMA. As formulated, the solution for the augmented design matrix is not unique. We consider several strategies using ideas from experimental design to create the augmented design matrix.

Keywords:

Our application deals with the assessment and propagation of uncertainty in a complex computer traffic model. We approach the estimation/propagation of uncertainty in the data through a probabilistic network. We augment the parameter set with latent counts representing number of cars at each location in the system. Sampling the 200-dimensional space requires dealing with a posterior subject to 37 linear restrictions. These restrictions come from direct observation of some car counts and the physical structure of the network. We find a direct way to reparameterize the posterior in one simple step. The reparameterized posterior has an additional problem of iteration-dependent support for most full conditionals. We show a way to exactly compute the support in one single computation per iteration and full conditional. A third problem is to find non-negative, integer-valued starting values for 127 latent counts subject to the 37 restrictions. We identify sufficient conditions to avoid dealing with the Diophantine problem of finding starting values, providing a direct way to generate starting values. The output of our MCMC is used as input to the complex computer model. We provide a comparison of results before and after uncertainty propagation.

Keywords:

We present two Bayesian approaches for sampling from the posterior distribution of a contingency table having a fixed set of marginal totals. Both of these methods are based on data augmentation and Bayesian model averaging. However, they are essentially different because one of them makes use of Markov bases (Diaconis and Sturmfels, 1998) to generate a feasible table consistent with the fixed marginals, while the other employs a novel sequential cell sampling algorithm described in Dobra, Tebaldi and West (2002). We illustrate the advantages/disadvantages of using these two methods by examples.

Keywords: Spatial-temporal modeling; Covariance selection; Decomposable graphical model; Empirical finance; MCMC

We interpret the classical portfolio selection problem (Markowitz, 1952) as a covariance selection problem (Dempster, 1972). Individual stock or industry portfolio (log) returns can thus be jointly modeled as graphical Gaussian models. We consider the case when the underlying graphs are allowed to vary, albeit slowly, over time. Given a sequence of past realizations, our goal is to learn/predict the graphical structure underlying a present/future (multivariate) observation. The challenge is to fit models that take into account both the uncertainty about and the time-variation of the underlying graphical structure. We restrict attention only to decomposable graphical models, which enjoy a factorization property that allows for local calculation of likelihood ratios in the evaluation of MCMC proposals. Using well-established algorithmic criteria for edge deletion and addition (Giudici and Green, 1999), a carefully designed proposal strategy allows for a relatively expeditious exploration of the state space.

Keywords: Cointegrated models, hypothesis test, bootstrap test, Bartlett correction.

In this paper we consider computer intensive methods to inference on cointegrating vectors in maximum likelihood analysis. In the first part of this paper we propose bootstrapping Johansen’s LR test for linear restriction on the cointegrating space and we compare the finite sample properties of the asymptotic, the bootstrap, and the bootstrap Bartlett corrected likelihood ratio test. Our Monte Carlo results show that asymptotic Chi-square based inference is quite inaccurate for sample sizes that are usually available in practical applications. By contrast, the Monte Carlo evaluation of the bootstrap and the bootstrap Bartlett corrected LR tests delivers remarkably accurate inference for the restrictions considered. Furthermore, the Monte Carlo evaluation of the power reveals that the power of the bootstrap, and bootstrap Bartlett corrected likelihood is almost as good as the asymptotic power, although in some situations the bootstrap Bartlett corrected LR test shows higher power than the bootstrap test. In the second part of this paper we propose bootstrapping the Bartlett corrected likelihood ratio test, but in this case the Bartlett correction is calculated analytically using the correction factor proposed by Johansen (2000). According to theoretical arguments proposed by Beran (1988) this procedure produces an error of rejecting probability of order O(T^-2), which is considerably small than the conventional first order approximation. The simulation results reveal that the bootstrapped procedure work remarkably well, although the response surface analysis reveals that the size distortion of the test heavily depends on the parameters space.

Keywords: Monte Carlo simulation; mixture of discrete distributions; fractals; immune response

To develop a theoretical framework for the quantification of the immune response to influenza virus, we considered modeling the generation the T cell clones (clonotypes) specific to a viral peptide. We introduce: 1) a set of new sequential Monte Carlo algorithms for simulating the dynamic process of the immune response; 2) a set of potential scenarios developed for the initiation and propagation of the immune response; 3) an assessment of a model selection procedure based on information measures; and 4) a novel graphical presentation for modeling using fractal self-similar polygonal spirals. Entropy measure assessment was performed for two modeling approaches: one by approximating actual data by a mixture of discrete distributions ("data-to-model" approach) and another by modeling a dynamic birth-death process ("model-to-data" approach). For visualization and further analysis of a clonotype distribution, we employed a recursive algorithm of constructing a set of spirals by rescaling, rotating, and shifting an origin, allowing a spiral to branch out. By applying the rank-frequency summary and the proposed modeling approach to a clonotype distribution, we discovered the self-similar fractal biphasic nature of an immune response. The use of Monte Carlo simulation according to the proposed set of potential scenarios for the initiation and propagation of the anti-viral response demonstrates potential pathways of generating heavy tails in a clonotype distribution. Visualization of the actual and simulated data using self-similar polygonal spirals possesses the valuable features of interpretability and sufficiency in mirroring the complexity of the fractal immune responses.

Keywords: Bayesian model averaging, shrinkage, model selection

Natural sounds can be meaningfully represented as a superposition of translated and frequency-modulated versions of simple functions (Gabor atoms). Moreover, representative features of such signals, along with prior knowledge concerning their generative mechanisms, may be conveniently and saliently described in the time-frequency plane through the use of Gabor frames. Here, after constructing Bayesian models and prior distributions capable of taking into account the time-frequency characteristics of typical audio waveforms, we apply Markov chain Monte Carlo methods in order to sample from the resultant posterior distributions of interest. We then demonstrate, by way of real-world examples, the potential of these methods in applications such as audio signal enhancement, compression, and modelling.

Keywords: MCMC, multilevel sampling, sensitivity analysis

A common problem affecting the convergence of MCMC algorithms is the presence of a (possibly unknown) number of separated posterior modes, which cannot be easily visited in the appropriate proportion by standard techniques such as the Gibbs sampler. We present and develop an approach for simultaneously attacking posterior multimodality and providing sensitivity analysis to prior specification. An example drawn from the case report delay distribution in AIDS surveillance illustrates our technique.

Keywords:

In the context of evaluating complex computer models, the problem of Bayesian analysis involving Gaussian spatial processes, with partially unknown mean and covariance structures, is encountered. To carry out needed McMC posterior computations, a special product form of the covariance function is considered. This, together with use of a design strategy that leads to at least some of the variables being observed in a Cartesian product fashion, allows for use of a Kronecker product formulation of the covariance matrix to considerably speed up the calculations involved in the sampling mechanism. For this problem, we devise and compare two McMC strategies, based on the ability to calculate the maximum likelihood estimates and the availability of a closed form expression for both the integrated likelihood and the Fisher information matrix.

Keywords: Markov chain Monte Carlo; Metropolis-Hastings algorithm; adaptive rejection sampling; auxiliary variables; penalized spline; spatial lattice data

We study the fitting of Bayesian generalized linear mixed models in full generality, rather than special cases geared towards grouped data. Such general design generalized linear mixed models allow for the usual analyses of longitudinal and spatial lattice data, but also include generalized additive models, generalized kriging and various combinations of these. We focus on the Bayesian analyses of general design generalized linear mixed models, and explore the use of three different Markov chain Monte Carlo (MCMC) approaches: Metropolis-Hastings algorithm, Adaptive Rejection Sampling method, and auxiliary variables within Gibbs sampling. We also consider hierarchical centering in the implementation of the MCMC schemes. The performance of each technique in inference of the general design generalized linear mixed models is compared using simulation studies, as well as some real data.

Keywords: Dirichlet Process, Clustering, Gibbs Sampler, Hierarchical Model

A common prior distribution used for modeling an unknown distribution is the Dirichlet process. A consequence of using the Dirichlet process is that the posterior distribution will be discrete, which is often cited as a weakness of this type of model. However, this discrete attribute can be utilized as a means of clustering observations. This model can be implemented via a Gibbs sampling algorithm and can easily be nested within a hierarchical structure for the modeling of a variety of different statistical problems. Thus, the use of a Dirichlet process is an interesting alternative to other well-established clustering methods. An application to the clustering of protein-binding motifs is presented, along with a comparison between the Dirichlet process and other potential prior distributions.

Keywords: Metropolis-Hastings, spectral gap, control variates

This paper proposes a control variate method for reducing the variance of an expectation via a simulation through independent Metropolis-Hastings algorithm. Assume that $X_0$ is distributed according the target distribution, $X_1,\ldots,X_n$ is the sample set generated from the Markov chain and Q is the auxiliary distribution used in the M-H algorithm. Let $Y_1,\ldots,Y_n$ be the random variables generated from Q. Our estimate of $E[h(X_0)]$ is given by $(1/n)\sum (h(X_i)-b g(Y_i))$ where $g$ is the covariate and $b$ is the regression constant. We show that $b=Cov(h(X_0),g(X_0))+ \sqrt{Var(h(X_0))Var(g(X_0))}{\lambda\over 1-\lambda}O(1/n)$ where $1-\lambda$ is the spectral gap. We also provide simalation results.

Keywords: Bayes, On-line Learning, Sequential Monte Carlo, Rao-Blackwellisation, Reconstructions and Predictions of Nonlinear Dynamical Systems

Sequential Monte Carlo (SMC) is a powerful sampling based inference/learning algorithm for Bayesian scheme.The purpose of this paper is two fold. It first attempts to reconstruct and predict nonlinear dynamical systems from one dimensional data which arrives in a sequential manner instaed of batch manner. Second purpose is to test the performance of the Rao-Blackwellisation in constructing and predicting nonlinear dynamical systems. We demonstrate Rao-Blackwellised Sequential Monte Carlo (RBSMC) on a chaotic time series prediction and reconstruction problem comparing to standard SMC.

Keywords: Perfect sampling, Simulated tempering, Rejection sampling, Disease mapping, Markov chain Monte Carlo, Bayesian inference

Spatial Poisson models using conditional autoregressions are commonly used in Bayesian modeling of areal data. Inference for such models is generally carried out via Markov chain Monte Carlo methods. In this paper, we propose two methods for producing independent samples from the posterior distribution. The first is based on marginal rejection sampling of the variance components using heavy tailed proposal distributions. Our second method takes advantage of our rejection sampling envelopes to produce a perfect sampling scheme for these models, along the lines of the perfect sampling algorithm described in Moller and Nicholls (1999). We describe the application of our methods to Minnesota cancer data sets.

Keywords: Protein simulation, Lattice polymer, monte carlo method, packing defects, voids in protein

Packing defects such as voids exist in all proteins and are often associated with protein functions. We study the statistical geometry of voids in two-dimensional lattice with various side chain models. We investigate how side chain affect the formation of void and their statistical properties in lattice polymer under models with side chain length 1-3, with either fixed or flexible orientation. For short chain polymers we perform exhaustive enumeration to gain a comprehensive understanding of the properties. For long chain polymers, sequential Monte Carlo (SMC) importance sampling and resampling techniques were used to explore the large conformational space that can not be exhaustively searched by enumeration. We characterize the relationship of geometric properties of voids with chain length, including probability of void formation, expected number of voids, void size and wall size of voids. Because proteins are not perfectly packed and therefore do not have conformations of maximum compactness, we focus on near-compact conformations (NCC) defined as conformations having 70%-90% of their maximum contact numbers. We found these conformations contain rich information on voids. Methods to generate NCCs using SMC are presented. Using NCCs we studied the side chain's effect for the formation of void during the folding of chain polymers for both linear chain model and side chain model. We also studied the effect of side chain to packing density.

Keywords: particle filtering, Bayesian dynamic models,nonlinear state-space models, prosthetics, spike-trains

Recent advances in technology have made it possible to monitor the activity of multiple neurons in a localized portion of an animal's brain. In particular, so-called ``spike-trains'' can be recorded simultaneously for multiple neurons. Neuroscientists have used this technology to model the relationship between spike-trains recorded from the motor cortex region of a monkey's brain and the motion of the monkey's arm, while the monkey carries out trained tasks. This has led to the development of ``population vector'' methods which predict the motion of the monkeys arm, based only on observation of the spike-trains. The fact that this can be done holds great promise for the development of prosthetic devices which could be controlled directly from the (human) brain. We discuss Bayesian dynamic models relating monkey neuron spike-trains to arm motion, and we show how particle filtering methods can be used to determine posterior distributions of arm movements, given observed spike-trains. We also show that under certain assumptions, these methods significantly out-perform the standard population vector methods in terms of predictive accuracy.

Keywords: Evolutionary Monte Carlo, Mutation, Crossover, Exchange, Metropolis-Hastings Algorithm, Markov Chain Monte Carlo.

Liang and Wong (JASA, 2001, 653-66) have proposed the Real-Parameter Evolutionary Monte Carlo (Real EMC) algorithm. This multiple chain sampling technique is shown to work better than the existing parallel tempering method. The Real EMC scheme comprises of many types of moves one of which is called the ``exchange'' move. We propose a few new types of exchange strategies which have very intuitive appeal. These moves on the one hand help force the ``good'' samples come down the temperature ladder and on the other the ``bad'' ones are forced to climb up the ladder. The theoretical properties of these kinds of moves are still under investigation but empirically we observe that these moves reduce the auto-correlation considerably in most of the cases and thereby improve the performance of the algorithm.

Keywords: Bayesian cokriging, spatio-temporal modeling, Kronecker structure, SCDFs

A spatial cumulative distribution function (SCDF) gives the proportion of a spatial domain D having the value of some response variable less than a particular level w. We model SCDFs via a hierarchical Bayesian approach (implemented by MCMC simulation), as discussed in Dr. Bradley Carlin's presentation. SCDFs may be weighted by covariates in several ways; in particular by assuming a bivariate random process setting, which allows simultaneous modeling of both the responses and the weights. MCMC methods (combined with a convenient Kronecker structure) enable straightforward univariate weighted and bivariate SCDF estimates, as well as joint and conditional SCDF estimates. We illustrate with an air pollution data set (publicly available from the California Air Resources Board) that measures NO and NO_2 levels on July 6, 1999. Often, air pollution measurements are collected at few locations but at multiple times; thus we may be able to obtain better precision by including a temporal component in our models. We present some initial results for extending the SCDFs to a spatio-temporal setting, again using a Kronecker structure to model covariance in the cokriging step. The model also allows for data that is missing completely at random. We illustrate with an air pollution data set that measures ozone and PM_{10} levels over a 12 month period in Vancouver, BC.

Keywords: Factor, point process, birth-death, prior, hyperparameter, posterior

In the study of the factor analysis model, two of the most common goals are the determination of the number of factors and the derivation of a "simple" structure. The classical approach deals with these two tasks separately, and often resorts to ad-hoc methods. This paper adopts a Bayesian approach to those problems, and adapts ideas from stochastic geometry and Bayesian finite mixture modelling to construct an ergodic Markov chain having the posterior distribution of the complete collection of parameters (including the number of factors) as its equilibrium distribution. The algorithm proposed combines a Gibbs sampler updating scheme with the discrete simulation of a continuous-time birth-and-death point process to produce a sampling scheme that efficiently explores the posterior distribution of interest. The MCMC sample path obtained from the simulated posterior then provides a flexible ingredient for most of the inferential tasks of interest. Illustrations on both artificial and real tasks are provided, while challenges and ideas for future improvements are also presented.

Keywords: Data Augmentation; Gene regulation; Missing data; Transcription factor binding sites

The Gibbs sampler (GS) has provided a feasible and technically sound computational alternative (to several existing deterministic methods) for finding an unknown number of short, partially conserved, repetitive patterns of unknown composition in sequences of symbols. An important scenario where this question arises is in the detection of transcription factor binding sites in DNA sequences. However, the GS may sometimes (i) fail to detect the presence of multiple types of patterns that may be present in the data, or (ii) fail to distinguish a true longer pattern in the data in the presence of long repeats of a single symbol or a collection of a few symbols, which is often the case when the successive sequence symbols (in the "background") are not generated independently. We present a novel data augmentation methodology for progressively detecting multiple unknown patterns under the framework of a stochastic ``dictionary'' model, where individual symbols and patterns are treated as stochastic ``words''. Examples are also presented to illustrate the improved performance of this algorithm for sequences generated under background models of varying levels of complexity.

Keywords:

This article proposes the skewed link model for analyzing ordinal response data with covariates which is based on Chen, Dey and Shao(1999). Using a Bayesian approach, we first investigate the propriety of posterior distrbutions using a uniform improper prior distribution on the regression parameters. We examine the proposed method through a simulation study. Then Markov chain Monte Carlo(MCMC) method is employed in order to avoid the complicated Bayesian computation. To accelerate MCMC convergence, the reparametrized MCMC is used which was used in Nandam and Chen(1996). Our limited experience shows that the proposed algorithm provides fast convergence.

Keywords: sequential monte carlo, Kalman filter, nonlinear

In this paper we present several new sequential Monte Carlo (SMC) algorithms for online estimation (filtering) of nonlinear dynamic systems. SMC has been shown to be a powerful tool for dealing with complex dynamic systems. It sequentially generates Monte Carlo samples from a {\it proposal distribution}, adjusted by a set of importance weight with repect to a target distribution, to facilitate statistical inferences on the charateristic (state) of the system. The key to a successful implementation of SMC in complex problems is the design of an efficient proposal distribution from which the Monte Carlo samples are generated. We propose several such proposal distributions that are efficient yet easy to generate samples from. They are efficient because they tend to utilize both the information in the state process and the observations. They are all Gaussian distributions hence are easy to sample from. The central ideas of the conventional nonlinear filters, such as extended Kalman filter, uncented Kalman filter and the Gaussian quadrature filter, are used to construct these proposal distributions. The effectiveness of the proposed algorithms are demonstrated through two applications -- real time target tracking and the multiuser parameter tracking in CDMA communication systems.

Keywords: Bayesian model average, Sequential Monte Carlo, Reversible jump Markov Chain Monte Carlo

Recently, an adaptive Bayesian receiver for blind detection in flat-fading channels was developed by Chen, Wang and Liu (2000), based on the sequential Monte Carlo methodology. That work is built on a parametric modelling of the fading process in the form of a state-space model, and assumes the knowledge of the second-order statistics of the fading channel. In this paper, we develop a nonparametric approach to the problem of blind detection in fading channels, without assuming any knowledge of the channel statistics. The basic idea is to decompose the fading process using a wavelet basis, and to use the sequential Monte Carlo technique to track both the wavelet coefficients and the transmitted symbols. However, conditioning on the single selected number of wavelet coefficients ignores the number uncertainty in the approximation of fading process, thus leads to the underestimation of uncertainty when making decision on the transmitted symbol. A reversible jump Markov Chain Monte Carlo (rjMCMC) based Bayesian model averaging (BMA) is proposed to deal with the number uncertainty. Under such a framework, blind detectors for flat-fading channels are developed. Simulation results are provided to demonstrate the excellent performance of the proposed blind adaptive receivers.

Keywords: Network tomography, Markov Chain Monte Carlo, EM

Network tomography is the technique that infer the network characteristics from the reports of end-to-end measurements of probing packets. In this paper, we present a Bayesian solution for network tomography via Monte Carlo Markov Chain (MCMC) methods. We firstly formulate Gibbs sampler for the inference of mean loss rates when the receiver reports full measurements of probing packets at any time or only the amounts of packets that are delivered; the latter is called noninvasive measurements since it does not need extra protocol to implement. Most existing works on loss inference focus on capturing the mean loss; but for the same mean loss rate, different loss patterns can produce different perceptions of QoS. To infer the bursty pattern at internal links, we further propose a modified Gibbs sampler based on the Gilbert model. The proposed Bayesian inference method is composed of a G-step to sample the internal loss function at internal links and a M-step to find the maximum likelihood estimation of the model parameter based on the samples from previous G-step. To our best knowledge, this is the first paper to investigate the bursty pattern inference problem conditioned on limited end-to-end measurements. Finally, our simulation results show the good performance of our proposed Bayesian inference methods.

Keywords: Sequential Monte Carlo method, Mixture Kalman filter, Multilevel sampling

Recently, mixture Kalman filter (Chen and Liu, 2001), a special sequential Monte Carlo methods, has shown it's powerful properties in solving online filtering of nonlinear dynamic system model, especially the conditional dynamic linear model (CDLM). The mixture Kalman filter is to draw samples of indicator variables in a recursive sequential importance sampling (SIS) way, while integrating out the linear and Gaussian state variables conditioning on the indicator variables. The performance of sequential Monte Carlo filtering can be substantially improved by this method. Due to the marginalization process, the complexity of mixture Kalman filter is impractical if the dimension of indicator sampling space is quite high. In this paper, we address this difficulty by developing a new Monte Carlo sampling scheme, namely, multilevel mixture Kalman filter. The basic idea is to make use of the multilevel structure, also hierarchical structure, of the indicator random variable. That is, we draw samples in a multilevel way, beginning with sampling from the highest-level sampling space and then draw samples from associate subspace of the newly drawn sample in a lower-level sampling space, until to the desired sampling space. The multilevel sampling can be used in conjunction with the delayed estimation method, such as the the delayed sample algorithm, resulting in a delayed multilevel sampling. Examples in digital communications are given to show the proposed sequential Monte Carlo filter.

Keywords: Battlespace, danger potential field, particle filter, resampling, Scaled Unscented Transformation, Sequential Importance Sampler

Assimilating data in a highly changing dynamic battlespace, which allows battle commanders to make timely, informed decisions, is a difficult and challenging problem. To approach this problem, a spatial-temporal statistical approach to examining the battlespace is taken, based on noisy data from multiple signals. We examine the danger-potential field (or danger field) generated by the positions of an enemy's weapons in the battlespace. The incoming noisy data is filtered to update the weapons' positions and the danger field. Sequential Importance Sampling, sometimes known as particle filtering, is used to generate realizations from the posterior distribution of the spatial-temporal danger field. Based on these realizations, non-linear questions such as the locations of maximum and minimum danger, the extent of regions exceeding certain danger thresholds, and changes in the danger field over time can be addressed. One particular Sequential Importance Sampler, known as the Unscented particle filter, is compared to faster but less-accurate approximations based on the Kalman filter, using data generated from an object-oriented combat-simulation program.

Keywords: isotonic regression; Gibbs sampler; posterior predictive distribution; Deviance Information Criterion

Isotonic regression (IR) is of interest in many fields (e.g., statistics, operations research, image processing, psychology) because it hypothesizes qualitative order constraints, and thus provides a more flexible alternative to ordinary regression that hypothesizes linear (additive) constraints. IR, successfully developed in the paradigm of classical (frequentist) statistics, is limited in applicability for at least three reasons. First, parameter estimation time exponentially increases with the number of IR model parameters. Second, the sampling (null) distribution is in general not available for testing the fit of IR models in detail. And third, it is difficult to measure the complexity of a given IR model, useful for performing model selection. This poster presents a general Bayes framework for isotonic regression, which addresses these three important issues. First, the framework is entirely based on a general Gibbs sampling algorithm that can estimate the posterior distribution of any given IR model. In fact, this algorithm consists of only two stages (per iteration), no matter how many parameters are contained in the model. Second, by interpreting the posterior predictive distribution, the framework is able to evaluate any aspect of data fit for any given IR model. Finally, with the posterior-based, Deviance Information Criterion (DIC), it routinely measures complexity of a given IR model, and performs model selection between different IR models. As a demonstration, the Bayes framework is presented and applied to analyze a data set generated by a national 6-item reading exam taken by 3,000 grade-school students.

Keywords: Spatial Interactions, Space-Time Modeling, Ocean Sedimentation

We propose a superclass of Gaussian spatial interaction models that includes as special cases CAR and SAR - like models. The class contains two spatial parameters, one of which quantifies the smoothness of the spatial random field. This class is comparable to the Matern class in geostatistics, but inherits the computational advantages of interaction models. Furthermore we choose a parametrization that requires minimal computation for likelihood and posterior calculations. The model is particularly suited as a component in hierarchical space time models for large environmental applications: Here it greatly helps to accelerate computations within each step of the MCMC based posterior calculations We apply this model to a data of ocean colors from the Sea-WiFS satellite with the goal of relating the space-time field to water discharge series from the dominant rivers.

Keywords: Interval censoring, Kaplan-Meyer technique, Monte Carlo estimates

Estimation of the average cost and utilization of medical services in the treatment of patients with a particular disease, in a patient sample with variable follow-up intervals leads to the problem of so-called "interval censoring". Straightforward implementation of the Kaplan-Meyer technique for survival analysis, widely cited in contemporary literature, is not completely applicable for such data. Results are compared to more complicated methods for dealing with interval censored data, when densities are not sampled directly, by simulation of randomly chosen time intervals as well as on real financial data on a population of patients with Diabetes and Benign Prostatic Hyperplasia.

Keywords:

I will describe a software system written in Java and designed to be extensible enough to facilitate analysis of any new statistical model using Markov chain Monte Carlo. This system relieves users of the responsibility of evaluating full conditional distributions and of selecting priors based on their convenience and conjugacy properties. The system provides a versatile alphabet for specifying models, and makes it simple to start a chain running whose updates are based on componentwise random walk Metropolis steps. In our experience, this default approach is adequate for many problems, and if it fails, it does so in predictable ways that can be solved using the system's techniques for simultaneous Metropolis updating of multiple parameters. We illustrate the system's power with several novel examples: a hierarchical model of mixture distributions for software reliability that requires reversible jump MCMC; a model for reliability of a system with test data and/or expert opinion related to components, subsystems, and the entire system; and a sports example with an unusual likelihood function and over 1500 parameters. The system inspires some questions about convergence acceleration for MCMC that are of little interest outside the system but if answered in the affirmative would make the system still more useful.

Keywords: Adaptive Metropolis-Hastings, Newton's Method, Hessian, multiple-try Metropolis

In this paper, we propose two efficient Markov chain Monte-Carlo sampling methods, namely, Hessian-based Metropolis-Hastings (HMH) and adaptive multiple importance-try (AMIT) algorithms. HMH utilizes Newton's optimization method to generate transition distributions in a Metropolis-Hastings sampling scheme. By introducing learning rates into Newton's method, HMH reduces the risk of having samples stuck in a local region where it is not good to approximate the target distribution by a Gaussian. We also provide a method to efficiently implement HMH for high-dimensional data. For more effective exploration of the adaptive proposal distributions obtained by HMH, AMIT combines HMH with multiple-try Metropolis (MTM) algorithm, proposed by Liu et al~\cite{LiuLiaWon2000}. We compare HMH and AMIT samplers with Gibbs and optimal marginal data augmentation (DA) samplers, proposed by van Dyk and Meng~\cite{vanDykMeng01}, on a probit regression problem. In the experiment, AMIT sampler outperform other samplers. Though only tested on a probit model, these new sampling methods can be easily applied to any generalized linear model and other models for which we can efficiently compute or approximate Hessian matrices.

Keywords: data assimilation, high-dimensional dynamical systems, ensemble Kalman filter, numerical weather prediction

We present efficient sample based approximations to the problem of sequentially estimating and tracking atmospheric states for numerical weather prediction. In a geophysical context, state estimation has formidable obstacles: the dimension of the state-vector in most oceanic and atmospheric models is extremely high, often exceeding $10^6$ components, and the systems are patently nonlinear, leading to non-Gaussian pdfs. In this work, we consider efficient Monte-Carlo approximations to the state-estimation problem for high-dimensional nonlinear systems using particle-filter ideas and localization of the state vector and observations. The presented method extends the ensemble Kalman filter using mixtures, and represents local covariance structures using nearest neighbors. The resulting algorithm, referred to as a mixture ensemble Kalman filter (XEnsF), is shown to be superior to existing methods in simulations on a low-dimensional model. The mixture filter also scales to high-dimensional systems by limiting (localizing) the impact of observations on estimates of the state vector. A second algorithm, referred to as a Local-local ensemble filter (LLEnsF), sequentially updates the state of the system using localizations in both phase space as well as physical space. This filter is shown to be stable and produces superior results when local non-Gaussian structures are present in the forecast sample

Keywords:

Financial models for asset and derivatives pricing, risk management, portfolio optimization, and asset allocation rely on volatility forecasts. Time-varying volatility models, such as GARCH and Stochastic Volatility (SVOL), have been successful in improving forecasts over constant volatility models. We develop a multivariate SVOL framework for modeling financial data that assumes covariance matrices stochastically varying through a Wishart random walk. Model fitting is performed using Markov chain Monte Carlo simulation from the posterior distribution. Due to the complexity of the model, an efficiently designed Gibbs sampler is necessary to produce inferences with a manageable amount of computation. The details of the Gibbs sampler implementation is discussed. Our approach is demonstrated on a multivariate time series of monthly industry portfolio returns.

Keywords:

Microarray data are generally known to suffer from contamination with many sources of systematic error. The process of removing these systematic effects is referred to as 'normalization'. Here, we discuss some of the statistical problems that occur in normalization and methods to handle these problems. A method based on robust statistical methodologies is described. The properties of the propposed method based on the M-estimators and L-estimators are discussed.

Keywords:

We review recent extended ensemble approaches that depend on temperature scaling or adjustment of the energy distribution to escape low energy traps. Examples of such methods include simulated and parallel tempering, dynamic weighting, multicanonical sampling and evolutionary Monte Carlo. We suggest that the use of a population of micro-canonical distributions may lead to further improvement.

Keywords: test

This is a test