Package 'DeCovarT'

Title: Covariance-Aware Deconvolution of Bulk Transcriptomic Samples
Description: Estimates cell-type proportions in bulk transcriptomic samples with a probabilistic convolution model that integrates the gene-gene covariance structure of purified reference profiles. Cellular ratios are recovered by maximum likelihood under a multivariate Gaussian convolution, using analytic gradients and Hessians, an isometric log-ratio reparametrisation (Helmert basis) that enforces the simplex constraint, and Marquardt-Levenberg or Newton-type optimisers. The methodology is described in Chassagnol, Nuel and Becht (2023) <doi:10.48550/arXiv.2309.09557>.
Authors: Bastien Chassagnol [aut, cre, cph] (ORCID: <https://orcid.org/0000-0002-8955-2391>)
Maintainer: Bastien Chassagnol <[email protected]>
License: MIT + file LICENSE
Version: 2.3.1
Built: 2026-10-22 22:52:44 UTC
Source: https://github.com/bastienchassagnol/DeCovarT

Help Index


Algorithm-similarity correlation from a Monte Carlo benchmark

Description

Pearson correlations rab=cor(p^a,p^b)r_{ab}=\mathrm{cor}(\hat p_a,\hat p_b) across Monte Carlo replicates (and cell types). This is behavioural similarity: two solvers can correlate near 1 while remaining systematically biased. Hierarchical order uses Ward D2 on dab=1−rabd_{ab} = 1 - r_{ab}.

Usage

algorithm_similarity(benchmark, facet_rows = NULL, facet_cols = NULL)

Arguments

benchmark

List from run_simulation_benchmark().

facet_rows, facet_cols

Optional scenario columns. When supplied, correlations are computed inside each facet cell.

Value

A tibble with algorithm_x, algorithm_y, correlation, and any facet columns.

See Also

plot_algorithm_similarity(), pivot_mc_estimates()

Examples

set.seed(1)
genes <- paste0("g", 1:2)
cts <- paste0("ct", 1:2)
mu <- matrix(c(20, 22, 22, 20), nrow = 2, dimnames = list(genes, cts))
Sigma <- array(
  c(1, 0, 0, 1, 1, 0, 0, 1),
  dim = c(2, 2, 2),
  dimnames = list(genes, genes, cts)
)
out <- run_simulation_benchmark(
  tibble::tibble(
    true_theta = list(list(p = c(0.5, 0.5), mu = mu, sigma = Sigma))
  ),
  deconvolution_functions = list(
    "nnls" = list(FUN = deconvolute_ratios_nnls),
    "rlm" = list(FUN = deconvolute_ratios_rlm)
  ),
  n = 4,
  cores = 1
)
algorithm_similarity(out)

Parametric bootstrap for DeCovarT proportions and boundary tests

Description

The conditional law of the bulk profile is fully specified, so the asymptotic calibrations of lrt_decovart() can be replaced by simulation. Given plug-in moments and a composition p\boldsymbol{p}, the function draws Y(b)∼NG(μp,Σ(p))\boldsymbol{Y}^{(b)}\sim \mathcal{N}_{G}(\boldsymbol{\mu}\boldsymbol{p}, \boldsymbol{\Sigma}(\boldsymbol{p})), refits DeCovarT, and returns the empirical distribution of p^(b)\hat{\boldsymbol{p}}^{(b)} together with percentile intervals.

Usage

bootstrap_decovart(
  y,
  mean_signature_matrix,
  Sigma,
  p = NULL,
  null_value = NULL,
  n_boot = 500L,
  level = 0.95,
  n_replicates = NULL,
  epsilon = 10^-8,
  itmax = 500L
)

Arguments

y

Numeric vector of length GG, or G×NG\times N matrix of replicate bulk profiles that share one composition; their log-likelihoods are summed (the pooled, population-level fit).

mean_signature_matrix

Numeric matrix μ∈MG×J\boldsymbol{\mu}\in\mathcal{M}_{G\times J} (plug-in means).

Sigma

Array of cell-type covariances in MG×G×J\mathcal{M}_{G\times G\times J}.

p

Composition to simulate from. Defaults to the MLE computed from y (or the restricted MLE under null_value).

null_value

Optional numeric vector of hypothesised ratios named by cell types; switches to the restricted bootstrap (see Details).

n_boot

Number of bootstrap replicates.

level

Confidence level for percentile intervals.

n_replicates

Number of bulk columns simulated per replicate; defaults to the number of observed columns.

epsilon, itmax

Relative convergence tolerance and iteration budget passed to stats::nlminb().

Details

Supplying null_value switches to a restricted bootstrap: replicates are generated under the restricted MLE and the likelihood-ratio statistic is recomputed on each, giving a Monte Carlo p-value that does not rely on the 50:5050{:}50 asymptotic weights. For a composite null the remaining nuisance proportions are profiled at their restricted estimate, so the procedure is a plug-in (approximate) rather than an exact bootstrap.

Reference uncertainty is not propagated here: μ\boldsymbol{\mu} and Σj\boldsymbol{\Sigma}_j are treated as known, exactly as in the frequentist likelihood. To resample the purified observations that produced those plug-in moments, use reference_bootstrap_decovart().

Value

A list with estimates (J×J\times n_boot matrix), interval (percentile bounds), n_failed (refits that errored, left as NA columns), p_simulated, and, when null_value is supplied, statistic (observed DD), null_statistics, and p_value.

See Also

lrt_decovart(), chi_bar_square_pvalue(), reference_bootstrap_decovart()

Other decovart_inference: bootstrap_single_cell(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_decovart(), confint_ilr_backtransform(), confint_profile_decovart(), confint_profile_single_cell(), confint_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), fit_firth_single_cell(), lrt_decovart(), lrt_single_cell(), multistart_decovart(), reference_bootstrap_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

mu <- matrix(c(20, 40, 15, 40, 20, 25), nrow = 3)
colnames(mu) <- paste0("ct", 1:2)
Sigma <- array(c(diag(3), diag(3)), dim = c(3, 3, 2))
y <- drop(mu %*% c(0.6, 0.4))
set.seed(1)
bootstrap_decovart(y, mu, Sigma, n_boot = 20)$interval

Parametric bootstrap for one single-cell bulk sample

Description

Draws Y(b)∼N(c^μ~p^,ϕ^SW(p^))\boldsymbol{Y}^{(b)}\sim\mathcal{N}(\hat c\tilde{\boldsymbol{\mu}} \hat{\boldsymbol{p}},\hat\phi\boldsymbol{S}_W(\hat{\boldsymbol{p}})) and refits the profile model. Percentile intervals are returned for the composition and for the two scales. The reference moments stay fixed.

Usage

bootstrap_single_cell(
  y,
  mean_signature_matrix,
  Sigma,
  n_boot = 200L,
  level = 0.95,
  epsilon = 1e-06,
  itmax = 40L
)

Arguments

y

Bulk vector of length GG.

mean_signature_matrix

Reference mean, G×JG\times J.

Sigma

Cell-level covariances, G×G×JG\times G\times J.

n_boot

Number of replicates.

level

Confidence level.

epsilon

Relative function and step tolerances passed to stats::nlminb().

itmax

Maximum number of Newton iterations.

Value

A list with estimates, interval, n_failed (refits that errored, left as NA columns), scales, scale_interval and fit.

See Also

confint_single_cell(), bootstrap_decovart()

Other decovart_inference: bootstrap_decovart(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_decovart(), confint_ilr_backtransform(), confint_profile_decovart(), confint_profile_single_cell(), confint_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), fit_firth_single_cell(), lrt_decovart(), lrt_single_cell(), multistart_decovart(), reference_bootstrap_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

mu <- matrix(c(2, 0.2, 0.2, 2, 0.2, 3, 0.2, 3), nrow = 4)
colnames(mu) <- c("ct1", "ct2")
sigma <- array(rep(diag(0.05, 4), 2), c(4, 4, 2))
y <- 20 * drop(mu %*% c(0.7, 0.3)) + c(0.3, -0.2, 0.1, -0.1)
set.seed(1)
bootstrap_single_cell(y, mu, sigma, n_boot = 5)$interval

Chi-bar-square tail probability for boundary likelihood-ratio tests

Description

Upper-tail probability of the mixture

χˉ2=∑i=0qwi χs+i2,wi=(qi)2−q,\bar{\chi}^{2} = \sum_{i=0}^{q} w_i\,\chi^{2}_{s+i}, \qquad w_i=\binom{q}{i}2^{-q},

with qq constraints active on the boundary of the parameter space and ss interior constraints. This is Case 9 of (Self and Liang 1987), which generalises the 12χ02+12χ12\tfrac{1}{2}\chi^{2}_{0}+\tfrac{1}{2}\chi^{2}_{1} result of (Chernoff 1954). For q=0q=0 the mixture collapses to the ordinary χs2\chi^{2}_{s} of (Wilks 1938).

Usage

chi_bar_square_pvalue(
  statistic,
  n_boundary = 1L,
  df_interior = 0L,
  weights = NULL
)

Arguments

statistic

Numeric vector of observed likelihood-ratio statistics D=2(ℓ^1−ℓ^0)D=2(\hat{\ell}_1-\hat{\ell}_0).

n_boundary

Integer qq: number of constraints active on the boundary (e.g. null ratios set to zero).

df_interior

Integer ss: number of interior (two-sided) constraints tested simultaneously.

weights

Optional numeric vector of length n_boundary + 1 giving the mixing weights w0,…,wqw_0,\ldots,w_q. Defaults to the binomial weights.

Details

The binomial weights hold when the block of the Fisher information corresponding to the active constraints is diagonal after orthogonalisation against the nuisance block. Under correlated constraints the mixing weights depend on the geometry of the approximating tangent cone and the binomial choice is only an approximation; Self and Liang also exhibit a nuisance-parameter configuration whose limit is not a chi-square mixture. Supply weights explicitly, or calibrate by simulation with bootstrap_decovart(), whenever q>1q>1 and the constrained cell types share signal.

Value

Numeric vector of upper-tail probabilities.

Atom at zero

χ02\chi^{2}_{0} is a point mass at the origin, so its strict upper tail Pr⁡(χ02>D)\Pr(\chi^{2}_{0}>D) is zero for every D≥0D\ge 0. The component therefore only removes weight: with one active constraint and no interior constraint the p-value is 12Pr⁡(χ12>D)\tfrac{1}{2}\Pr(\chi^{2}_{1}>D), capped at 0.50.5 when D=0D=0. Note stats::pchisq() switches to the closed tail at exactly q = 0 for df = 0; the strict convention is used here so the p-value stays continuous in statistic.

References

Self SG, Liang K (1987). “Asymptotic Properties of Maximum Likelihood Estimators and Likelihood Ratio Tests under Nonstandard Conditions.” Journal of the American Statistical Association, 82(398), 605–610. ISSN 0162-1459. doi:10.1080/01621459.1987.10478472.

Chernoff H (1954). “On the Distribution of the Likelihood Ratio.” The Annals of Mathematical Statistics, 25(3), 573–578. ISSN 0003-4851. doi:10.1214/aoms/1177728725.

Wilks SS (1938). “The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses.” The Annals of Mathematical Statistics, 9(1), 60–62. ISSN 0003-4851. doi:10.1214/aoms/1177732360.

See Also

lrt_decovart()

Other decovart_inference: bootstrap_decovart(), bootstrap_single_cell(), boundary_diagnostics(), confint_decovart(), confint_ilr_backtransform(), confint_profile_decovart(), confint_profile_single_cell(), confint_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), fit_firth_single_cell(), lrt_decovart(), lrt_single_cell(), multistart_decovart(), reference_bootstrap_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

# One active boundary constraint: 50:50 mixture of chi2_0 and chi2_1.
chi_bar_square_pvalue(2.71, n_boundary = 1L)
# Interior (Wilks) calibration.
chi_bar_square_pvalue(3.84, n_boundary = 0L, df_interior = 1L)

One-dominant composition with a target normalised Shannon entropy

Description

Returns a length-JJ simplex vector of the form (1−(J−1)q, q,…,q)(1-(J-1)q,\,q,\ldots,q) whose normalised Shannon entropy compute_shannon_entropy() equals h_star. The uniform composition (h_star = 1) and a Dirac mass (h_star = 0) are returned exactly.

Usage

composition_from_entropy(h_star, n_celltypes = 3L, nms = NULL)

Arguments

h_star

Target H⋆∈[0,1]H^{\star}\in[0,1].

n_celltypes

Integer J≥2J\ge 2.

nms

Optional names for the returned vector.

Value

Numeric simplex vector of length JJ.

See Also

compute_shannon_entropy()

Examples

p <- composition_from_entropy(0.5, n_celltypes = 3L)
compute_shannon_entropy(p)

Average pairwise Jeffreys divergence of a Gaussian mixture

Description

Returns the pjpℓp_j p_{\ell}-weighted mean of pairwise Jeffreys (symmetrised KL) divergences between purified Gaussians, with the closed form of the multivariate-normal Kullback–Leibler divergence (https://statproofbook.github.io/P/mvn-kl.html)

J‾=∑1≤j<ℓ≤Jpjpℓ J(fj,fℓ)∑1≤j<ℓ≤Jpjpℓ∈[0,∞),\overline{J} = \frac{ \sum_{1\le j<\ell\le J} p_j p_{\ell}\, J(f_{j},f_{\ell}) }{ \sum_{1\le j<\ell\le J} p_j p_{\ell} } \in[0,\infty),

with fj=NG(μ⋅j,Σj)f_{j}=\mathcal{N}_{G}( \boldsymbol{\mu}_{\cdot j},\boldsymbol{\Sigma}_{j} ). If p is omitted it defaults to the equi-balanced vector 1/J1/J, which recovers the uniform pairwise average 2/(J(J−1))∑j<ℓJ(fj,fℓ)2/(J(J-1))\sum_{j<\ell}J(f_{j},f_{\ell}).

Usage

compute_average_jeffreys(true_theta, J = NULL)

Arguments

true_theta

List with the generative parameters (see .parse_true_theta()): mu (G×JG\times J), sigma (G×G×JG\times G\times J), and optionally p (length JJ or J×NJ\times N). If p is missing it is set to (1/J,…,1/J)(1/J,\ldots,1/J).

J

Number of cell types. Defaults to the third dimension of sigma.

Value

Scalar average pairwise Jeffreys divergence.

See Also

compute_average_overlap(), compute_average_riemannian()

Examples

set.seed(1)
theta <- list(
  mu = cbind(c(0, 0), c(3, 0)),
  sigma = array(c(diag(2), diag(2)), dim = c(2, 2, 2))
)
compute_average_jeffreys(theta)

Average pairwise overlap of a Gaussian mixture

Description

Returns MixSim's BarOmega: the unweighted mean of pairwise overlaps

ω‾=2J(J−1)∑1≤j<ℓ≤J(Ωjℓ+Ωℓj)∈[0,1]\overline{\omega} = \frac{2}{J(J-1)} \sum_{1\le j<\ell\le J} \bigl(\Omega_{j\ell}+\Omega_{\ell j}\bigr) \in[0,1]

(up to the MixSim numerical convention), where Ωjℓ=Pr⁡X∼fj(X classified as ℓ)\Omega_{j\ell}=\Pr_{X\sim f_j}(X\text{ classified as }\ell) already uses the mixture weights p\boldsymbol{p} inside the Bayes / MAP rule of MixSim::overlap(). Do not multiply the directional masses by pjp_j again.

For G≥4G\ge 4 MixSim's Davies quadrature becomes expensive. The helper then switches to overlap_gaussian_mc(): stratified Sobol draws, inverse-transform sampling through precomputed Cholesky factors, and log-density MAP comparisons (n_mc draws per component; default 10,000).

Usage

compute_average_overlap(
  true_theta,
  J = NULL,
  n_mc = 10000L,
  seed = NULL,
  verbose = TRUE
)

Arguments

true_theta

List with the generative parameters (see .parse_true_theta()): p (length JJ or J×NJ\times N), mu (G×JG\times J), sigma (G×G×JG\times G\times J).

J

Number of cell types (components). Defaults to the third dimension of sigma.

n_mc

Monte Carlo draws per component when G≥4G\ge 4.

seed

Optional seed forwarded to overlap_gaussian_mc().

verbose

If TRUE (default), announce the MixSim-to-MC switch with cli when it is installed.

Value

Scalar average pairwise overlap (MixSim BarOmega).

See Also

overlap_gaussian_mc(), compute_average_jeffreys()

Examples

set.seed(1)
theta <- list(
  p = c(0.5, 0.5),
  mu = cbind(c(0, 0), c(3, 0)),
  sigma = array(c(diag(2), diag(2)), dim = c(2, 2, 2))
)
compute_average_overlap(theta)

Mean pairwise AIRM distance of component covariances

Description

Average over j<ℓj<\ell of the affine-invariant Riemannian metric on the cone of symmetric positive-definite matrices, dR(Σj,Σℓ)=∥log⁡(Σj−1/2ΣℓΣj−1/2)∥Fd_R(\Sigma_j,\Sigma_\ell)=\lVert\log(\Sigma_j^{-1/2}\Sigma_\ell \Sigma_j^{-1/2})\rVert_F, computed from the eigenvalues of R−TΣℓR−1R^{-\mathsf{T}}\Sigma_\ell R^{-1} with Σj=RTR\Sigma_j=R^{\mathsf{T}}R. The Frobenius chord ∥Σj−Σℓ∥F\lVert\Sigma_j-\Sigma_\ell\rVert_F is not used: it leaves the cone, is not inversion-invariant, and suffers the swelling effect. With J=2J=2 the average is the pairwise distance.

Usage

compute_average_riemannian(true_theta, J = NULL)

Arguments

true_theta

List with the generative parameters (see .parse_true_theta()): mu (G×JG\times J), sigma (G×G×JG\times G\times J), and optionally p (length JJ or J×NJ\times N). If p is missing it is set to (1/J,…,1/J)(1/J,\ldots,1/J).

J

Number of cell types. Defaults to the third dimension of sigma.

Value

Scalar average of dR(Σj,Σℓ)d_R(\Sigma_j,\Sigma_\ell) over j<ℓj<\ell.

Examples

a <- diag(2)
b <- matrix(c(2, 0.5, 0.5, 1), nrow = 2)
theta <- list(
  mu = cbind(c(0, 0), c(1, 0)),
  sigma = array(c(a, b), dim = c(2, 2, 2))
)
compute_average_riemannian(theta)

Compute deconvolution benchmark metrics

Description

Returns three blocks: composition / regression scores, Monte Carlo summaries (ADEMP-style), and optimisation / runtime diagnostics. When p⋆\boldsymbol{p}^{\star} is a matrix J×NJ\times N (or p^\hat{\boldsymbol{p}} is), cell-type Pearson correlation and Monte Carlo summaries are computed across samples, not within a single composition.

Usage

compute_benchmark_metrics(
  y,
  mean_signature_matrix,
  estimated_p,
  true_ratios = NULL,
  se = NULL,
  elapsed = NULL,
  memory_bytes = NULL,
  kkt_residual = NULL,
  numerical_converged = NULL,
  loglik_hat = NULL,
  loglik_true = NULL,
  converged = NULL,
  local_maximum = NULL,
  min_eigenvalue = NULL,
  max_eigenvalue = NULL,
  presence_threshold = 1e-04,
  level = 0.95,
  coverage_interval = "wilson",
  algorithm = NA_character_
)

Arguments

y

Numeric vector (or one-column matrix) y∈RG\boldsymbol{y}\in\mathbb{R}^{G}.

mean_signature_matrix

Numeric matrix μ∈MG×J\boldsymbol{\mu}\in\mathcal{M}_{G\times J} (plug-in means).

estimated_p

Estimated proportions: a length-JJ vector or a numeric matrix J×NJ\times N.

true_ratios

Optional ground truth with the same layout. When omitted, reconstitution scores compare y^=μp^\hat{\boldsymbol{y}}=\boldsymbol{\mu}\hat{\boldsymbol{p}} with y\boldsymbol{y}.

se

Optional standard errors matching estimated_p. Convolution-likelihood solvers fill these from vcov_ilr_delta() (expected Fisher information pulled back through the ILR chart). Mean-only solvers (NNLS, LSEI) leave them missing.

elapsed, memory_bytes, kkt_residual

Optional per-sample runtime diagnostics (recycled to NN).

numerical_converged

Optional logical per sample: the solver returned a finite simplex vector.

loglik_hat, loglik_true

Optional per-sample log-likelihoods used for theoretical convergence (regret ℓ(p⋆)−ℓ(p^)\ell(\boldsymbol{p}^{\star})-\ell(\hat{\boldsymbol{p}})).

converged

Optional list of length NN of optimiser records with fields iterations, criterion, code and rdm, as stored by the DeCovarT solvers. Closed-form solvers store NA entries.

local_maximum

Optional logical per sample: ILR score small and Hessian negative definite (λmin⁡<0\lambda_{\min}<0 and λmax⁡<0\lambda_{\max}<0).

min_eigenvalue, max_eigenvalue

Optional ILR Hessian extrema.

presence_threshold

Threshold ε\varepsilon for presence / F1 / false-positive mass (default 1e-4).

level

Level of the Wald intervals built from se, used for coverage and mean width (default 0.95).

coverage_interval

Interval for the Monte Carlo coverage rate π^\hat\pi: "wilson" (default), "wald", or "agresti_coull". This is not the interval for pjp_j itself.

algorithm

Character label stored on each metrics row (typically the solver name). Recycled to length one.

Value

A named list with:

  • regression: global (one row per sample) and cell_type (one row per cell type);

  • monte_carlo: one row per cell type, including theoretical_se (filled later from expected Fisher at p⋆p^{\star});

  • optimisation: one row per sample.

Examples

mu <- matrix(c(20, 22, 22, 20), nrow = 2,
             dimnames = list(paste0("g", 1:2), paste0("ct", 1:2)))
y <- drop(mu %*% c(0.4, 0.6))
compute_benchmark_metrics(y, mu, estimated_p = c(0.45, 0.55),
                          true_ratios = c(0.4, 0.6))

Normalised Shannon entropy of a discrete distribution

Description

For a probability vector p∈ΔJ−1\boldsymbol{p}\in\Delta^{J-1} over JJ classes (here: cell types), the Shannon entropy is

H(p)=−∑j=1Jpjlog⁡pj,H(\boldsymbol{p}) = -\sum_{j=1}^{J} p_j \log p_j,

with the convention 0log⁡0=00\log 0 = 0. Dividing by the maximum entropy log⁡J\log J (uniform over all JJ classes) yields normalised Shannon entropy

H⋆(p)=H(p)log⁡J∈[0,1],H^{\star}(\boldsymbol{p}) = \frac{H(\boldsymbol{p})}{\log J} \in[0,1],

so H⋆=0H^{\star}=0 for a Dirac mass on one type and H⋆=1H^{\star}=1 for the uniform distribution over the JJ cell types. Zero masses are dropped only inside the sum; the normaliser still uses the original class count JJ.

This is preferable to changing the logarithm base to the number of positive masses J′J', which would renormalise only on the support and hide sparsity relative to the full panel of cell types.

Usage

compute_shannon_entropy(ratios)

Arguments

ratios

Numeric vector of cellular proportions p∈ΔJ−1\boldsymbol{p}\in\Delta^{J-1} (length JJ).

Value

Scalar normalised entropy H⋆∈[0,1]H^{\star}\in[0,1].

Illustration

Panel B of the figure contrasts a peaked (type-specific, low entropy) share vector with a flat (multi-type, high entropy) one for J=5J=5 classes.

Gini versus Shannon entropy

See Also

composition_from_entropy()

Examples

compute_shannon_entropy(c(1, 0, 0)) # Dirac -> 0
compute_shannon_entropy(rep(1 / 3, 3)) # uniform -> 1

Choose a confidence interval for a pooled DeCovarT composition

Description

One entry point for the interior intervals of the population convolution (covariance weights pj2p_j^2, no technical scales). The default, method = "ilr", is confint_ilr_backtransform().

ilr

Wald interval for ρ\boldsymbol{\rho}, mapped by the inverse ILR chart.

wald

Delta method on p\boldsymbol{p}. Endpoints can leave [0,1][0,1].

firth

Wald interval at the Jeffreys-penalised estimate (fit_firth_decovart()). interval_ilr is the same estimate mapped through the inverse chart.

profile

Likelihood-ratio interval (confint_profile_decovart()). This is the interior calibration. A null on a face is lrt_decovart(), not an interval.

bootstrap

Parametric percentile interval (bootstrap_decovart()).

reference

Resample the reference, then refit (reference_bootstrap_decovart()). Two or more donors in every cell type use the donor bootstrap and keep the bulk fixed. Otherwise columns are resampled within each cell type and the bulk is regenerated. Cell-type labels are not shuffled; that check is equivariance_check_decovart().

Columns of y share one composition. fit_decovart() instead estimates a separate composition per column, and confint.decovart_fit() is the per-column delta method.

Usage

confint_decovart(
  y,
  mean_signature_matrix = NULL,
  Sigma = NULL,
  method = c("ilr", "wald", "firth", "profile", "bootstrap", "reference"),
  level = 0.95,
  p = NULL,
  n_draws = 2000L,
  n_boot = 500L,
  seed = NULL,
  reference_profiles = NULL,
  donor_ids = NULL,
  resample = NULL,
  regenerate_bulk = NULL,
  ...
)

Arguments

y

Numeric vector of length GG, or G×NG\times N matrix of replicate bulk profiles that share one composition; their log-likelihoods are summed (the pooled, population-level fit).

mean_signature_matrix

Numeric matrix μ∈MG×J\boldsymbol{\mu}\in\mathcal{M}_{G\times J} (plug-in means).

Sigma

Array of cell-type covariances in MG×G×J\mathcal{M}_{G\times G\times J}.

method

Interval to compute. See Description.

level

Confidence level.

p

Composition at which to evaluate the information. NULL uses the pooled maximum likelihood estimate.

n_draws

Monte Carlo size used when there are three or more cell types. Ignored for two cell types.

n_boot

Bootstrap size for method = "bootstrap" or "reference".

seed

Optional seed for the Monte Carlo map. The previous random-number state is restored. NULL leaves the state unchanged and draws from the current stream.

reference_profiles, donor_ids

Passed to reference_bootstrap_decovart() when method = "reference".

resample

"donors" or "cells". NULL chooses from donor_ids as in Description.

regenerate_bulk

Whether the reference bootstrap redraws the bulk. NULL follows the automatic rule in Description.

...

Passed to the selected method (epsilon, itmax, celltypes, null_value, ridge, n_replicates).

Value

A list with method, estimate and interval. Other elements depend on method.

See Also

lrt_decovart(), confint_single_cell()

Other decovart_inference: bootstrap_decovart(), bootstrap_single_cell(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_ilr_backtransform(), confint_profile_decovart(), confint_profile_single_cell(), confint_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), fit_firth_single_cell(), lrt_decovart(), lrt_single_cell(), multistart_decovart(), reference_bootstrap_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

mu <- matrix(c(20, 40, 15, 40, 20, 25), nrow = 3)
colnames(mu) <- paste0("ct", 1:2)
Sigma <- array(c(diag(3), diag(3)), dim = c(3, 3, 2))
y <- drop(mu %*% c(0.6, 0.4))
confint_decovart(y, mu, Sigma)$interval
confint_decovart(y, mu, Sigma, method = "wald")$interval

Confidence intervals by mapping a Wald region through the ILR chart

Description

Forms a Wald interval for the unconstrained ILR coordinate ρ\boldsymbol{\rho} from the expected Fisher information, then maps it to the simplex by the inverse chart ψ\boldsymbol{\psi}. For two cell types the map is monotone, so the endpoints of the interval for ρ\rho are transformed. For more than two cell types the joint normal draw ρ(b)\boldsymbol{\rho}^{(b)} is transformed and the interval is the marginal quantile of ψ(ρ(b))\boldsymbol{\psi}(\boldsymbol{\rho}^{(b)}). Every transformed draw is a composition, so the interval lies in (0,1)(0,1).

This is the default of confint_decovart(). The symmetric delta-method interval on p\boldsymbol{p} remains confint.decovart_fit(). Columns of y are replicates that share one composition: their number multiplies the Fisher information.

Usage

confint_ilr_backtransform(
  y,
  mean_signature_matrix,
  Sigma,
  p = NULL,
  level = 0.95,
  n_draws = 2000L,
  seed = NULL,
  epsilon = 10^-8,
  itmax = 500L
)

Arguments

y

Numeric vector of length GG, or G×NG\times N matrix of replicate bulk profiles that share one composition; their log-likelihoods are summed (the pooled, population-level fit).

mean_signature_matrix

Numeric matrix μ∈MG×J\boldsymbol{\mu}\in\mathcal{M}_{G\times J} (plug-in means).

Sigma

Array of cell-type covariances in MG×G×J\mathcal{M}_{G\times G\times J}.

p

Composition at which to evaluate the information. NULL uses the pooled maximum likelihood estimate.

level

Confidence level.

n_draws

Monte Carlo size used when there are three or more cell types. Ignored for two cell types.

seed

Optional seed for the Monte Carlo map. The previous random-number state is restored. NULL leaves the state unchanged and draws from the current stream.

epsilon, itmax

Relative convergence tolerance and iteration budget passed to stats::nlminb().

Value

A list with estimate, interval on the simplex, rho, rho_interval (component-wise Wald bounds for ρ\boldsymbol{\rho}), vcov_rho, vcov, mapping ("endpoint" or "monte_carlo") and n_obs. On a simplex face the interval is NA.

See Also

confint_decovart(), vcov_ilr_delta(), confint.decovart_fit()

Other decovart_inference: bootstrap_decovart(), bootstrap_single_cell(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_decovart(), confint_profile_decovart(), confint_profile_single_cell(), confint_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), fit_firth_single_cell(), lrt_decovart(), lrt_single_cell(), multistart_decovart(), reference_bootstrap_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

mu <- matrix(c(20, 40, 15, 40, 20, 25), nrow = 3)
colnames(mu) <- paste0("ct", 1:2)
Sigma <- array(c(diag(3), diag(3)), dim = c(3, 3, 2))
y <- drop(mu %*% c(0.6, 0.4))
confint_ilr_backtransform(y, mu, Sigma)$interval

Profile-likelihood confidence intervals for cellular ratios

Description

Inverts the likelihood-ratio test of lrt_decovart() coordinate by coordinate: the interval collects every cc with 2{ℓ(p^)−ℓjprof(c)}≤χ1,1−α22\{\ell(\hat{\boldsymbol{p}})-\ell^{\mathrm{prof}}_j(c)\} \le\chi^{2}_{1,1-\alpha}. Because likelihood ratios are invariant under reparametrisation, the interval respects 0≤pj≤10\le p_j\le 1 without the delta-method linearisation that makes Wald intervals unreliable near a simplex face.

Usage

confint_profile_decovart(
  y,
  mean_signature_matrix,
  Sigma,
  level = 0.95,
  celltypes = colnames(mean_signature_matrix),
  epsilon = 10^-8,
  itmax = 500L
)

Arguments

y

Numeric vector of length GG, or G×NG\times N matrix of replicate bulk profiles that share one composition; their log-likelihoods are summed (the pooled, population-level fit).

mean_signature_matrix

Numeric matrix μ∈MG×J\boldsymbol{\mu}\in\mathcal{M}_{G\times J} (plug-in means).

Sigma

Array of cell-type covariances in MG×G×J\mathcal{M}_{G\times G\times J}.

level

Confidence level 1−α1-\alpha.

celltypes

Names of the cell types to profile (columns of mean_signature_matrix); defaults to all.

epsilon, itmax

Relative convergence tolerance and iteration budget passed to stats::nlminb().

Value

A matrix with one row per cell type and columns estimate, lower, upper.

See Also

confint.decovart_fit() for the Wald analogue, lrt_decovart()

Other decovart_inference: bootstrap_decovart(), bootstrap_single_cell(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_decovart(), confint_ilr_backtransform(), confint_profile_single_cell(), confint_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), fit_firth_single_cell(), lrt_decovart(), lrt_single_cell(), multistart_decovart(), reference_bootstrap_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

mu <- matrix(c(20, 40, 15, 40, 20, 25), nrow = 3)
colnames(mu) <- paste0("ct", 1:2)
Sigma <- array(c(diag(3), diag(3)), dim = c(3, 3, 2))
y <- drop(mu %*% c(0.6, 0.4))
confint_profile_decovart(y, mu, Sigma)

Profile-likelihood intervals for one single-cell bulk sample

Description

Inverts the likelihood-ratio test for one coordinate of the profile log-likelihood. The two scales are concentrated out at every trial value. An endpoint that lands on a face is not a Wilks interval; use lrt_single_cell() for a boundary null.

Usage

confint_profile_single_cell(
  y,
  mean_signature_matrix,
  Sigma,
  level = 0.95,
  celltypes = colnames(mean_signature_matrix),
  variance = c("ml", "reml"),
  epsilon = 1e-08,
  itmax = 100L
)

Arguments

y

Bulk vector of length GG.

mean_signature_matrix

Reference mean, G×JG\times J.

Sigma

Cell-level covariances, G×G×JG\times G\times J.

level

Confidence level.

celltypes

Cell types to profile. Defaults to every column.

variance

"ml" (default) uses the χ12\chi^2_1 cut-off. "reml" uses Glog⁡{1+F1,ν;1−α/ν}G\log\{1+F_{1,\nu;1-\alpha}/\nu\} with ν=G−J\nu=G-J, the exact cut-off of the profile likelihood ratio when a variance scale is profiled out of a linear model.

epsilon

Relative function and step tolerances passed to stats::nlminb().

itmax

Maximum number of Newton iterations.

Value

A matrix with columns estimate, lower and upper.

See Also

confint_single_cell(), lrt_single_cell()

Other decovart_inference: bootstrap_decovart(), bootstrap_single_cell(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_decovart(), confint_ilr_backtransform(), confint_profile_decovart(), confint_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), fit_firth_single_cell(), lrt_decovart(), lrt_single_cell(), multistart_decovart(), reference_bootstrap_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

mu <- matrix(c(2, 0.2, 0.2, 2, 0.2, 3, 0.2, 3), nrow = 4)
colnames(mu) <- c("ct1", "ct2")
sigma <- array(rep(diag(0.05, 4), 2), c(4, 4, 2))
y <- 20 * drop(mu %*% c(0.7, 0.3)) + c(0.3, -0.2, 0.1, -0.1)
confint_profile_single_cell(y, mu, sigma)

Confidence intervals for one single-cell bulk sample

Description

The composition and the two technical scales are separated by the Schur complement of the observed Hessian. method chooses the interval for the composition. The default is the inverse ILR chart applied to a Wald interval for ρ\boldsymbol{\rho}, using the profile information {−∇ρ2ℓ∗}−1\{-\nabla^{2}_{\boldsymbol{\rho}}\ell^{\ast}\}^{-1}.

Intervals for (c,ϕ)(c,\phi) in nuisance$interval are Wald intervals on the log scale, mapped back by the exponential, from the Schur complement that removes ρ\boldsymbol{\rho}. nuisance$interval_wald is the symmetric normal interval on the original scale. Bootstrap methods replace both with percentile intervals.

ilr

Default. Inverse ILR chart.

wald

Delta method on the composition. Endpoints can leave [0,1][0,1].

firth

Wald interval at the penalised profile estimate. interval_ilr maps that estimate through the inverse chart.

profile

Interior likelihood-ratio interval. A face null is lrt_single_cell().

bootstrap

Parametric percentile interval.

reference

Donor bootstrap, or a within-type cell resample that regenerates the bulk. See reference_bootstrap_single_cell().

These intervals treat the reference covariance as known. They do not add a sandwich or higher-order correction for that plug-in.

variance = "reml" corrects for the ν=G−J\nu=G-J residual degrees of freedom of the variance scale: covariances are inflated by G/νG/\nu, Wald and ILR quantiles come from tνt_{\nu} (a multivariate tt for the ILR map when J>2J>2), ϕ\phi gets the chi-square interval [Gϕ^/χν;1−α/22,Gϕ^/χν;α/22][G\hat\phi/\chi^2_{\nu;1-\alpha/2}, G\hat\phi/\chi^2_{\nu;\alpha/2}], and the profile interval uses the cut-off of confint_profile_single_cell(). The maximum-likelihood ϕ^\hat\phi is biased by about ν/G\nu/G, so with few genes (G=10G=10, J=2J=2) the "ml" intervals cover about 0.85 to 0.92 at the 0.95 level; the bootstrap methods are unaffected by variance.

Usage

confint_single_cell(
  y,
  mean_signature_matrix = NULL,
  Sigma = NULL,
  method = c("ilr", "wald", "firth", "profile", "bootstrap", "reference"),
  level = 0.95,
  variance = c("ml", "reml"),
  n_draws = 2000L,
  n_boot = 200L,
  seed = NULL,
  reference_profiles = NULL,
  donor_ids = NULL,
  resample = NULL,
  regenerate_bulk = NULL,
  ...
)

Arguments

y

Bulk vector of length GG.

mean_signature_matrix

Reference mean, G×JG\times J.

Sigma

Cell-level covariances, G×G×JG\times G\times J.

method

Interval for the composition. See Description.

level

Confidence level.

variance

"ml" (default) or "reml"; see Description.

n_draws

Monte Carlo size for the ILR map when J>2J>2.

n_boot

Bootstrap size.

seed

Optional seed, restored on exit, for the ILR Monte Carlo map and for the bootstrap methods.

reference_profiles, donor_ids, resample, regenerate_bulk

Passed to reference_bootstrap_single_cell() when method = "reference".

...

epsilon and itmax for the selected method.

Value

A list with method, estimate (the fitted composition), interval and nuisance. Bootstrap results also include the replicate matrices.

See Also

vcov_single_cell(), confint_decovart(), lrt_single_cell()

Other decovart_inference: bootstrap_decovart(), bootstrap_single_cell(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_decovart(), confint_ilr_backtransform(), confint_profile_decovart(), confint_profile_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), fit_firth_single_cell(), lrt_decovart(), lrt_single_cell(), multistart_decovart(), reference_bootstrap_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

mu <- matrix(c(2, 0.2, 0.2, 2, 0.2, 3, 0.2, 3), nrow = 4)
colnames(mu) <- c("ct1", "ct2")
sigma <- array(rep(diag(0.05, 4), 2), c(4, 4, 2))
y <- 20 * drop(mu %*% c(0.7, 0.3)) + c(0.3, -0.2, 0.1, -0.1)
fit <- confint_single_cell(y, mu, sigma)
fit$interval
fit$nuisance$interval

Monte Carlo interval for an empirical coverage probability

Description

Interval for a binomial coverage rate π^=X/N\hat\pi=X/N from independent coverage indicators. Wilson is the default because the parameter is bounded in [0,1][0,1] and Wald intervals behave poorly near 0 or 1.

Usage

coverage_mc_interval(
  covered,
  conf_level = 0.95,
  method = c("wilson", "wald", "agresti_coull")
)

Arguments

covered

Logical or 0/1 coverage indicators (one per replicate).

conf_level

Confidence level for the interval around π^\hat\pi (default 0.95).

method

"wilson" (default), "wald", or "agresti_coull".

Value

A list with n, successes, coverage, mcse, lower, upper, and method.

Examples

coverage_mc_interval(c(TRUE, TRUE, TRUE, FALSE))

Parallel deconvolution of a bulk expression matrix

Description

For each column y⋅i\boldsymbol{y}_{\cdot i} of the bulk matrix Y∈MG×N\boldsymbol{Y}\in\mathcal{M}_{G\times N}, estimates p^⋅i\hat{\boldsymbol{p}}_{\cdot i} with every supplied algorithm. Samples are iterated with furrr when cores > 1; algorithms are sequential, so workers are never nested. When covariance information is provided, DeCovarT methods maximise ℓy ∣ ζ(p)\ell_{\boldsymbol{y}\,|\,\boldsymbol{\zeta}}(\boldsymbol{p}) under y ∣ (ζ,p)∼NG(μp,Σ(p))\boldsymbol{y}\,|\,(\boldsymbol{\zeta},\boldsymbol{p})\sim \mathcal{N}_{G}( \boldsymbol{\mu}\boldsymbol{p}, \boldsymbol{\Sigma}(\boldsymbol{p}) ).

Usage

deconvolute_ratios(
  signature_matrix,
  bulk_expression,
  true_ratios = NULL,
  Sigma = NULL,
  deconvolution_functions = NULL,
  standardise = FALSE,
  scaled = FALSE,
  cores = getOption("mc.cores", 1L),
  coverage_interval = "wilson",
  verbose = FALSE,
  progress_every = 10L
)

Arguments

signature_matrix

Mean signature μ∈MG×J\boldsymbol{\mu}\in\mathcal{M}_{G\times J} (rownames = genes, colnames = cell types).

bulk_expression

Bulk matrix Y∈MG×N\boldsymbol{Y}\in\mathcal{M}_{G\times N} (rownames = genes, colnames = samples).

true_ratios

Optional ground-truth proportions: a length-JJ numeric vector (recycled over samples) or a numeric matrix J×NJ\times N (or N×JN\times J).

Sigma

Optional array (Σj)j∈MG×G×J(\boldsymbol{\Sigma}_j)_{j}\in\mathcal{M}_{G\times G\times J} of cell-type covariances (numeric; off-diagonals may be negative).

deconvolution_functions

Named list; each element has FUN and optional additional_parameters.

standardise

If TRUE, apply a gene-wise affine z-score computed once from μ\boldsymbol{\mu} to bulk, means and covariances (see Details). Cell-type-wise or sample-wise transforms are not supported.

scaled

Deprecated. TRUE (log2 mixing) always errors.

cores

Number of furrr workers for the sample loop. Defaults to getOption("mc.cores", 1L). Use cores = 1 to stay sequential (CRAN examples and nested callers). With cores > 1, .map_samples() uses furrr_options(seed = TRUE), which assigns an independent L'Ecuyer-CMRG stream per worker (R 4.1 / future streams). Do not offset a shared seed by the sample index.

coverage_interval

Passed to compute_benchmark_metrics() ("wilson", "wald", or "agresti_coull").

verbose

If TRUE, emit progress for sequential sample loops (every progress_every samples). Parallel furrr runs log a one-line notice instead of per-sample ticks.

progress_every

Sample-progress interval when verbose is TRUE and cores = 1. Defaults to 10L.

Value

A named list from compute_benchmark_metrics() with regression (global and cell-type subtables), monte_carlo, and optimisation (per-sample elapsed time, memory, KKT residual, and p^\hat{\boldsymbol{p}}). First-generation solvers still call repair_simplex(); the three native DeCovarT maps (ILR or p/∑pp/\sum p) already lie on the simplex. When Sigma is supplied, Newton-Raphson, Marquardt-Levenberg, L-BFGS-B, BFGS (gradient), and simulated annealing attach ILR Wald standard errors from vcov_ilr_delta() so monte_carlo can report coverage, mean model SE, and the SE/SD ratio. Those SEs are the expected-Fisher delta-method bounds used by confint.decovart_fit(), not diagonal entries of the observed Hessian. Mean-only solvers (NNLS, LSEI) leave Wald columns missing: those estimators do not use the convolution likelihood.

References

Erkkilä T, Lehmusvaara S, Ruusuvuori P, Visakorpi T, Shmulevich I, Lähdesmäki H (2010). “Probabilistic Analysis of Gene Expression Measurements from Heterogeneous Tissues.” Bioinformatics, 26. doi:10.1093/bioinformatics/btq406. Ahn J, Yuan Y, Parmigiani G, Suraokar MB, Diao L, Wistuba II, Wang W (2013). “DeMix: Deconvolution for Mixed Cancer Transcriptomes Using Raw Measured Data.” Bioinformatics (Oxford, England), 29. doi:10.1093/bioinformatics/btt301. Wang Z, Cao S, Morris JS, Ahn J, Liu R, Tyekucheva S, Gao F, Li B, Lu W, Tang X, Wistuba II, Bowden M, Mucci L, Loda M, Parmigiani G, Holmes CC, Wang W (2018). “Transcriptome Deconvolution of Heterogeneous Tumor Samples with Immune Infiltration.” iScience, 9. doi:10.1016/j.isci.2018.10.028. Anghel CV, Quon G, Haider S, Nguyen F, Deshwar AG, Morris QD, Boutros PC (2015). “ISOpureR: An R Implementation of a Computational Purification Algorithm of Mixed Tumour Profiles.” BMC Bioinformatics, 16. doi:10.1186/s12859-015-0597-x. Ogundijo OE, Wang X (2017). “A Sequential Monte Carlo Approach to Gene Expression Deconvolution.” PLOS ONE, 12. doi:10.1371/journal.pone.0186167. Hafemeister C, Satija R (2019). “Normalization and variance stabilization of single-cell RNA-seq data using regularized negative binomial regression.” Genome Biology, 20(1), 296. doi:10.1186/s13059-019-1874-1. Chion M, Leroy A (2023). “A Bayesian Framework for Multivariate Differential Analysis Accounting for Missing Data.” arXiv. doi:10.48550/arxiv.2307.08975. Newman A, Liu C, Green M, Gentles A, Feng W, Xu Y, Hoang C, Diehn M, Alizadeh A (2015). “Robust Enumeration of Cell Subsets from Tissue Expression Profiles.” Nature methods, 12. doi:10.1038/nmeth.3337.

See Also

deconvolute_ratios_Marquardt_Levenberg()

Examples

set.seed(1)
genes <- paste0("g", 1:2)
cts <- paste0("ct", 1:2)
mu <- matrix(c(20, 22, 22, 20), nrow = 2, dimnames = list(genes, cts))
Sigma <- array(
  c(1, 0, 0, 1, 1, 0, 0, 1),
  dim = c(2, 2, 2),
  dimnames = list(genes, genes, cts)
)
Y <- simulate_bulk_mixture(mu, Sigma, p = c(0.5, 0.5), n = 2)$Y
deconvolute_ratios(
  signature_matrix = mu,
  bulk_expression = Y,
  true_ratios = c(0.5, 0.5),
  Sigma = Sigma,
  deconvolution_functions = list(
    "nnls" = list(FUN = deconvolute_ratios_nnls)
  ),
  cores = 1
)

DeCovarT MLE of cellular proportions for one bulk sample

Description

Estimates p^=arg⁡max⁡pℓy ∣ ζ(p)\hat{\boldsymbol{p}}=\arg\max_{\boldsymbol{p}} \ell_{\boldsymbol{y}\,|\,\boldsymbol{\zeta}}(\boldsymbol{p}) under the Gaussian convolution model

y ∣ (ζ,p)∼NG ⁣(μp, Σ(p)),Σ(p)=∑j=1Jpj2Σj,\boldsymbol{y}\,|\,(\boldsymbol{\zeta},\boldsymbol{p}) \sim\mathcal{N}_{G}\!\bigl(\boldsymbol{\mu}\boldsymbol{p},\, \boldsymbol{\Sigma}(\boldsymbol{p})\bigr), \qquad \boldsymbol{\Sigma}(\boldsymbol{p})=\sum_{j=1}^{J}p_j^{2}\boldsymbol{\Sigma}_j,

subject to the simplex constraint 1Tp=1\mathbf{1}^{\mathsf{T}}\boldsymbol{p}=1, p≥0\boldsymbol{p}\ge\mathbf{0}. Optimisation is performed in unconstrained ILR coordinates z∈RJ−1\boldsymbol{z}\in\mathbb{R}^{J-1} via p=softmax⁡(Vz)\boldsymbol{p}=\operatorname{softmax}(\mathbf{V}\boldsymbol{z}) (Marquardt–Levenberg default; see other methods below and vignette("theory-decovart-generative-model", package = "DeCovarT")).

Usage

deconvolute_ratios_cibersort(y, mean_signature_matrix)

deconvolute_ratios_rlm(y, mean_signature_matrix)

deconvolute_ratios_nnls(y, mean_signature_matrix)

deconvolute_ratios_deconrnaseq(y, mean_signature_matrix)

deconvolute_ratios_Marquardt_Levenberg(
  y,
  mean_signature_matrix,
  Sigma,
  epsilon = 10^-4,
  itmax = 200,
  return_model = FALSE,
  initial_p = NULL,
  dirichlet_alpha = 1
)

deconvolute_ratios_simulated_annealing(
  y,
  mean_signature_matrix,
  Sigma,
  epsilon = 10^-4,
  itmax = 200,
  return_model = FALSE,
  initial_p = NULL,
  dirichlet_alpha = 1
)

deconvolute_ratios_L_BFGS_B(
  y,
  mean_signature_matrix,
  Sigma,
  epsilon = 10^-4,
  itmax = 200,
  return_model = FALSE,
  initial_p = NULL,
  dirichlet_alpha = 1
)

deconvolute_ratios_Newton_Raphson(
  y,
  mean_signature_matrix,
  Sigma,
  epsilon = 10^-4,
  itmax = 200,
  return_model = FALSE,
  initial_p = NULL,
  dirichlet_alpha = 1
)

deconvolute_ratios_gradient_descent(
  y,
  mean_signature_matrix,
  Sigma,
  epsilon = 10^-4,
  itmax = 200,
  return_model = FALSE,
  initial_p = NULL,
  dirichlet_alpha = 1
)

Arguments

y

Numeric vector (or one-column matrix) y∈RG\boldsymbol{y}\in\mathbb{R}^{G}.

mean_signature_matrix

Numeric matrix μ∈MG×J\boldsymbol{\mu}\in\mathcal{M}_{G\times J} (plug-in means).

Sigma

Array of cell-type covariances in MG×G×J\mathcal{M}_{G\times G\times J}.

epsilon, itmax

Absolute convergence tolerance and maximum number of iterations for the optimiser (same roles as reltol / maxit in stats::optim()).

return_model

If TRUE, return a named list with coefficients, ILR coordinates, log-likelihood and optimiser diagnostics instead of the proportion vector.

initial_p

Optional start: NULL / "barycentre" (default equi-balanced), a length-JJ numeric vector, "dirichlet", or "qp" (mean-only simplex QP). See starting_simplex(). Starts on a simplex face are nudged into the interior so the ILR map is defined (ILR methods) and so L-BFGS-B does not start on a degenerate Σ(p)\boldsymbol{\Sigma}(\boldsymbol{p}).

dirichlet_alpha

Dirichlet concentration when initial_p = "dirichlet" (default 1: uniform on the simplex; α>1\alpha>1 centre-biased; α<1\alpha<1 face-biased). Independent draws are independent restarts; use multistart_decovart() to sequence several.

Details

Plug-in signature. Argument mean_signature_matrix is the mean μ\boldsymbol{\mu}, used as a proxy for the unobserved cell-type profiles x⋅j\boldsymbol{x}_{\cdot j}. This is the frequentist plug-in; recovering sample-specific latents is a Bayesian / MAP problem (.map_gaussian_convolution()).

Value

Named numeric vector p^\hat{\boldsymbol{p}} on the simplex (ILR methods), or that list when return_model = TRUE. Benchmark metrics are computed by deconvolute_ratios().

Functions

  • deconvolute_ratios_cibersort(): Linear baseline y^=μp^\hat{\boldsymbol{y}}=\boldsymbol{\mu}\hat{\boldsymbol{p}} via nu-SVR (CIBERSORT-style); no covariance prior is used.

  • deconvolute_ratios_rlm(): Robust linear model y≈μp\boldsymbol{y}\approx\boldsymbol{\mu}\boldsymbol{p} (MASS::rlm()), as in Monaco et al. (2019).

  • deconvolute_ratios_nnls(): Non-negative least squares for y≈μp\boldsymbol{y}\approx\boldsymbol{\mu}\boldsymbol{p} (nnls::nnls()), then simplex projection.

  • deconvolute_ratios_deconrnaseq(): Equality- and inequality-constrained least squares on the simplex (limSolve::lsei()), in the spirit of deconRNASeq.

  • deconvolute_ratios_simulated_annealing(): Simulated annealing on ILR coordinates z\boldsymbol{z} (stats::optim() with method = "SANN").

  • deconvolute_ratios_L_BFGS_B(): Box-constrained L-BFGS-B directly in p\boldsymbol{p} (stats::optim() method = "L-BFGS-B"). The box keeps each coordinate in [0,1][0,1]; the returned vector is closed by p/∑pp/\sum p (no repair_simplex() clipping). Wald standard errors still use the ILR expected-Fisher map vcov_ilr_delta(), not a p\boldsymbol{p}-space Hessian.

  • deconvolute_ratios_Newton_Raphson(): Newton–Raphson on ILR coordinates z\boldsymbol{z} with the analytic score and observed Hessian, safeguarded by the PORT trust region of stats::nlminb().

  • deconvolute_ratios_gradient_descent(): BFGS quasi-Newton ascent on ILR coordinates z\boldsymbol{z} (stats::optim() method = "BFGS").

References

Monaco G, Lee B, Xu W, Mustafah S, Hwang YY, Carré C, Burdin N, Visan L, Ceccarelli M, Poidinger M, Zippelius A, Pedro de Magalhães J, Larbi A (2019). “RNA-Seq Signatures Normalized by mRNA Abundance Allow Absolute Deconvolution of Human Immune Cell Types.” Cell Reports, 26. doi:10.1016/j.celrep.2019.01.041.

See Also

deconvolute_ratios(), isometric_logistic(), starting_simplex()

Examples

set.seed(1)
genes <- paste0("g", 1:2)
cts <- paste0("ct", 1:2)
mu <- matrix(c(20, 22, 22, 20), nrow = 2, dimnames = list(genes, cts))
Sigma <- array(
  c(1, 0, 0, 1, 1, 0, 0, 1),
  dim = c(2, 2, 2),
  dimnames = list(genes, genes, cts)
)
y <- drop(mu %*% c(0.6, 0.4) + rnorm(2, sd = 0.1))
deconvolute_ratios_Marquardt_Levenberg(y, mu, Sigma, itmax = 50)

Single-cell deconvolution of one bulk sample

Description

Fits one bulk sample from the linear single-cell mixture y∼N(cμ~p,ϕSW(p))\boldsymbol{y}\sim\mathcal{N}(c\tilde{\boldsymbol{\mu}}\boldsymbol{p}, \phi\boldsymbol{S}_W(\boldsymbol{p})).

The start is single_cell_nuisance_scales(): nonnegative least squares with a free scale (DeconRNASeq's quadratic programme without the simplex equality), then the closed-form scales at that composition.

With profile = TRUE (default) the two scales are profiled out: for every trial composition they are recomputed in closed form, so the weight matrix SW(p)−1\boldsymbol{S}_W(\boldsymbol{p})^{-1} is rebuilt at every iterate, and Newton-Raphson runs on the profile log-likelihood of the J−1J-1 ILR coordinates (profile_loglik_single_cell_ilr()). Its limit is the joint maximum-likelihood estimate of (p,c,ϕ)(\boldsymbol{p},c,\phi), reached in one loop.

With profile = FALSE the scales stay at their start values (a two-stage plug-in fit), which is not the joint maximum. Supplying both mean_scale and variance_scale fixes them at those values.

Every fit uses the analytic score and observed Hessian with the PORT trust region of stats::nlminb(): the step maximises the quadratic model inside a ball whose radius grows or shrinks with the ratio of actual to predicted increase; there is no line search and no fixed step length. The returned vcov is the delta-method image of the inverse negative profile Hessian; intervals are confint_single_cell().

Usage

deconvolute_single_cell(
  y,
  mean_signature_matrix,
  Sigma,
  epsilon = 1e-08,
  itmax = 100L,
  profile = TRUE,
  initial_p = NULL,
  mean_scale = NULL,
  variance_scale = NULL,
  estimand = c("cellular", "transcript"),
  cell_size = NULL
)

Arguments

y

Bulk vector of length GG.

mean_signature_matrix

Reference mean, G×JG\times J.

Sigma

Cell-level covariances, G×G×JG\times G\times J.

epsilon

Relative function and step tolerances passed to stats::nlminb().

itmax

Maximum number of Newton iterations.

profile

If TRUE, profile the two scales out; if FALSE, keep them at their start values.

initial_p

Optional open-simplex start. NULL uses the free-scale least-squares start.

mean_scale, variance_scale

Optional fixed scales. Supply both or neither.

estimand

cellular (default) interprets the fitted composition as a cell fraction, which is valid when the mean keeps the cell sizes SjS_j. transcript interprets it as a transcript fraction; pass cell_size to convert it back to a cell fraction.

cell_size

Optional SjS_j. Defaults to the column sums of the mean, which equal the cell sizes for a cellular_raw reference.

Value

A list with composition (the fitted simplex coordinate), p, q, rho, estimand, fit_type ("profile", "two_stage" or "fixed_scales"), mean_scale, variance_scale, platform_scale, n_cells, cell_size, ols_scale, loglik, convergence (0 from stats::nlminb() means success), iterations, p_start, message, vcov_rho, vcov and vcov_nuisance (see vcov_single_cell()). A singular information matrix is returned as NA without failing the fit.

See Also

single_cell_nuisance_scales(), profile_loglik_single_cell_ilr(), deconvolute_ratios_Newton_Raphson(), rescale_single_cell_moments()

Examples

genes <- paste0("g", 1:4)
cts <- paste0("ct", 1:2)
mu <- matrix(
  c(2, 2, 0.2, 0.2, 0.2, 0.2, 3, 3),
  nrow = 4,
  dimnames = list(genes, cts)
)
sigma <- array(0, c(4, 4, 2), dimnames = list(genes, genes, cts))
sigma[,, 1] <- diag(0.05, 4)
sigma[,, 2] <- diag(0.05, 4)
set.seed(1)
sim <- simulate_bulk_from_cells(
  mu,
  sigma,
  p = c(0.8, 0.2),
  n_cells = 2000,
  platform_scale = 1
)
fit <- deconvolute_single_cell(sim$Y[, 1], mu, sigma)
round(fit$p, 3)

Describe a Gaussian-convolution simulation scenario

Description

Summarises the generating law y∣(ζ,p)∼NG(μp,∑jpj2Σj)\boldsymbol{y}\mid(\boldsymbol{\zeta},\boldsymbol{p})\sim \mathcal{N}_{G}(\boldsymbol{\mu}\boldsymbol{p}, \sum_j p_j^2\boldsymbol{\Sigma}_j) at five layers: composition, mean geometry, covariance / SPD diagnostics, network sparsity, and tangent Fisher information (mean versus covariance split). MixSim BarOmega and average pairwise Hellinger of the component Gaussians are kept in descriptors (Hellinger is the unweighted pairwise mean; hellinger_weighted is optional). Jeffreys / symmetrised KL is returned in supplementary. Mean-geometry columns include the highest pairwise cosine and the mean Euclidean (CLD) gap.

Usage

describe_simulation_scenario(
  true_theta,
  adjacency = NULL,
  active_tol = 1e-08,
  include_mixsim = TRUE
)

Arguments

true_theta

List with p, mu, and sigma (and optionally Theta precision). Parsed by .parse_true_theta().

adjacency

Optional G×G×JG\times G\times J adjacency array, or a length-JJ list of adjacencies, used for network density when supplied.

active_tol

Threshold for counting an active simplex component.

include_mixsim

Logical. If TRUE (default), compute MixSim BarOmega via compute_average_overlap(). Set FALSE to skip.

Value

A list with:

  • theta_true: the convolution parameters p, mu, sigma;

  • descriptors: one-row tibble of kept scenario statistics in six families (composition, mean geometry, SPD of Σ(p)\boldsymbol{\Sigma}(\boldsymbol{p}), tangent Fisher, network, component overlap). MixSim BarOmega uses compute_average_overlap() (Davies quadrature for G<4G<4, Sobol Monte Carlo otherwise). Average pairwise Hellinger and affine-invariant Riemannian distance of the Σj\Sigma_j are kept alongside. hellinger_weighted is optional. SPD columns include both κ{Σ(p)}\kappa\{\boldsymbol{\Sigma}(\boldsymbol{p})\} (kappa_sigma_p) and the reciprocal λmin⁡/λmax⁡\lambda_{\min}/\lambda_{\max} (kappa_sigma_reciprocal);

  • supplementary: one-row tibble of Jeffreys / symmetrised KL, recorded but not treated as a primary score;

  • call: the matched call (match.call()).

See Also

run_simulation_benchmark(), expected_fisher_unconstrained(), compute_shannon_entropy(), composition_from_entropy(), helmert_basis(), compute_average_overlap(), compute_average_riemannian()

Examples

theta <- list(
  p = c(0.5, 0.5),
  mu = cbind(c(10, 0), c(0, 10)),
  sigma = array(c(diag(2), diag(2)), dim = c(2, 2, 2))
)
out <- describe_simulation_scenario(theta)
out$descriptors$h_star
out$call

Expected Fisher information of unconstrained p\boldsymbol{p}

Description

For the multivariate-normal mean–covariance map of the DeCovarT convolution, y∼NG(μp,Σ(p))\boldsymbol{y}\sim\mathcal{N}_{G}(\boldsymbol{\mu}\boldsymbol{p}, \boldsymbol{\Sigma}(\boldsymbol{p})) with Σ(p)=∑jpj2Σj\boldsymbol{\Sigma}(\boldsymbol{p})=\sum_j p_j^{2}\boldsymbol{\Sigma}_j and precision Ω(p)=Σ(p)−1\boldsymbol{\Omega}(\boldsymbol{p})=\boldsymbol{\Sigma}(\boldsymbol{p})^{-1}, the expected Fisher information has entries

I(p)jk=μ⋅j⊤Ω(p)μ⋅k+2pjpk tr(Ω(p)ΣjΩ(p)Σk).I(\boldsymbol{p})_{jk} = \boldsymbol{\mu}_{\cdot j}^{\top} \boldsymbol{\Omega}(\boldsymbol{p}) \boldsymbol{\mu}_{\cdot k} + 2 p_j p_k\, \mathrm{tr}\bigl( \boldsymbol{\Omega}(\boldsymbol{p})\boldsymbol{\Sigma}_j \boldsymbol{\Omega}(\boldsymbol{p})\boldsymbol{\Sigma}_k \bigr).

The first summand is the mean contribution (an Ω\boldsymbol{\Omega}-inner product of signature columns); the second is the covariance contribution of the quadratic map p↦Σ(p)\boldsymbol{p}\mapsto\boldsymbol{\Sigma}(\boldsymbol{p}). See the multivariate-normal formula on https://en.wikipedia.org/wiki/Fisher_information#Multivariate_normal_distribution.

Usage

expected_fisher_unconstrained(p, mean_signature_matrix, Sigma)

Arguments

p

Numeric proportions on the open simplex.

mean_signature_matrix

Mean signature μ\boldsymbol{\mu} (G×JG\times J).

Sigma

Cell-type covariances G×G×JG\times G\times J.

Value

Symmetric J×JJ\times J expected Fisher information matrix I(p)I(\boldsymbol{p}).

See Also

vcov_ilr_delta(), vcov.decovart_fit(), confint.decovart_fit(), hessian_loglik_unconstrained()

Examples

# Two cell types, bivariate Gaussians: I(p) is 2 x 2.
p <- c(0.6, 0.4)
mu <- cbind(c(0, 0), c(3, 0))
Sigma <- array(c(diag(2), diag(2)), dim = c(2, 2, 2))
I <- expected_fisher_unconstrained(p, mu, Sigma)
I

Fit the DeCovarT Gaussian-convolution model

Description

Maximum-likelihood estimator of cellular proportions p⋅i\boldsymbol{p}_{\cdot i} under the multivariate Gaussian convolution

y⋅i∣(μ,Σj,p⋅i)∼NG(μp⋅i,  Σ(p⋅i)),Σ(p)=∑j=1Jpj2Σj.\boldsymbol{y}_{\cdot i}\mid(\boldsymbol{\mu},\boldsymbol{\Sigma}_j, \boldsymbol{p}_{\cdot i}) \sim \mathcal{N}_{G}\bigl( \boldsymbol{\mu}\boldsymbol{p}_{\cdot i},\; \boldsymbol{\Sigma}(\boldsymbol{p}_{\cdot i}) \bigr), \qquad \boldsymbol{\Sigma}(\boldsymbol{p}) =\sum_{j=1}^{J}p_j^{2}\boldsymbol{\Sigma}_j.

This is a variance / likelihood specification, not ordinary least squares: cell-type covariances enter the residual law and cannot be written as extra columns of μ\boldsymbol{\mu}. There is therefore no formula / lm interface, and no predict() method for forecasting bulk expression.

The returned object is an S3 class decovart_fit with accessors coef(), fitted(), residuals(), vcov(), nobs(), confint(), print(), summary() and plot(). Residuals are Y−μP^\boldsymbol{Y}-\boldsymbol{\mu}\hat{\boldsymbol{P}}; they are not OLS residuals. Goodness of fit is the convolution log-likelihood, not residual sum of squares.

Usage

fit_decovart(
  signature_matrix,
  bulk_expression,
  true_ratios = NULL,
  Sigma = NULL,
  method = c("Newton-Raphson", "Marquardt-Levenberg", "L-BFGS-B"),
  epsilon = 10^-4,
  itmax = 200,
  standardise = FALSE,
  scaled = FALSE,
  n_starts = 0L,
  boundary_tol = 1e-08
)

## S3 method for class 'decovart_fit'
print(x, ...)

## S3 method for class 'decovart_fit'
summary(object, ...)

## S3 method for class 'summary.decovart_fit'
print(x, ...)

## S3 method for class 'decovart_fit'
coef(object, ...)

## S3 method for class 'decovart_fit'
fitted(object, ...)

## S3 method for class 'decovart_fit'
residuals(object, ...)

## S3 method for class 'decovart_fit'
vcov(object, ...)

## S3 method for class 'decovart_fit'
nobs(object, ...)

## S3 method for class 'decovart_fit'
confint(object, parm, level = 0.95, ...)

## S3 method for class 'decovart_fit'
plot(x, ...)

Arguments

signature_matrix

Mean signature μ∈MG×J\boldsymbol{\mu}\in\mathcal{M}_{G\times J} (rownames = genes, colnames = cell types).

bulk_expression

Bulk matrix Y∈MG×N\boldsymbol{Y}\in\mathcal{M}_{G\times N} (rownames = genes, colnames = samples).

true_ratios

Optional ground-truth proportions (JJ vector or J×NJ\times N matrix). Stored on the fit; not used by the MLE.

Sigma

Array (Σj)j∈MG×G×J(\boldsymbol{\Sigma}_j)_{j}\in\mathcal{M}_{G\times G\times J} of cell-type covariances.

method

Optimiser; one of "Newton-Raphson" (default), "Marquardt-Levenberg", "L-BFGS-B" (case-insensitive). These three maps already land on the simplex (ILR or p/∑pp/\sum p); they do not call repair_simplex().

epsilon, itmax

Absolute convergence tolerance and iteration budget, in the same roles as reltol / maxit in stats::optim().

standardise

If TRUE, apply a gene-wise affine z-score computed once from μ\boldsymbol{\mu} to bulk, means and covariances (see Details). Cell-type-wise or sample-wise transforms are not supported.

scaled

Deprecated. TRUE (log2 mixing) always errors.

n_starts

Number of additional random Dirichlet starts per sample. With n_starts > 0 the best log-likelihood is kept and multistart_decovart() records the spread of attained optima, the only direct probe of multimodality (the realised log-likelihood is not globally concave).

boundary_tol

Threshold on min⁡jp^j\min_j\hat{p}_j below which a sample is flagged near_boundary by boundary_diagnostics(). Wald intervals are unreliable there; prefer confint_profile_decovart() or lrt_decovart().

x, object

A decovart_fit.

...

Passed to graphics::plot() (plot method) or ignored.

parm

Kept for compatibility with stats::confint(); all simplex coordinates are always returned (subsetting by parm is not implemented). Removing this formal would break the S3 method contract with the generic.

level

Confidence level 1−α1-\alpha (default 0.95). Wald intervals use asymptotic normality of the MLE with standard errors from vcov_ilr_delta() / expected_fisher_unconstrained() (see Details of fit_decovart() and of vcov_ilr_delta()): p^j±z1−α/2 SEj\hat{p}_j \pm z_{1-\alpha/2}\,\mathrm{SE}_j.

Details

Standardisation. CIBERSORT requires non-negative expression, no missing values, and a non-log linear scale (Newman et al. 2015). A logarithm is concave, so Jensen's inequality shifts first and second moments and log⁡(μp)≠(log⁡μ)p\log(\boldsymbol{\mu}\boldsymbol{p})\neq(\log\boldsymbol{\mu})\boldsymbol{p}. A gene-wise affine map x↦D−1(x−m)\boldsymbol{x}\mapsto D^{-1}(\boldsymbol{x}-\boldsymbol{m}) with the same D,mD,\boldsymbol{m} on Y\boldsymbol{Y}, μ\boldsymbol{\mu} and Σj⋆=D−1ΣjD−1\boldsymbol{\Sigma}_j^\star=D^{-1}\boldsymbol{\Sigma}_j D^{-1} leaves the MLE of p\boldsymbol{p} unchanged (equivariance).

Wald covariance. Let Ω(p)=Σ(p)−1\boldsymbol{\Omega}(\boldsymbol{p})=\boldsymbol{\Sigma}(\boldsymbol{p})^{-1}. The expected Fisher information of the unconstrained mean–covariance map (multivariate normal; see e.g. the Wikipedia entry Fisher information, multivariate normal) is

I(p)jk=μ⋅j⊤Ω(p)μ⋅k+2pjpk tr(Ω(p)ΣjΩ(p)Σk).I(\boldsymbol{p})_{jk} = \boldsymbol{\mu}_{\cdot j}^{\top} \boldsymbol{\Omega}(\boldsymbol{p}) \boldsymbol{\mu}_{\cdot k} + 2 p_j p_k\, \mathrm{tr}\bigl( \boldsymbol{\Omega}(\boldsymbol{p})\boldsymbol{\Sigma}_j \boldsymbol{\Omega}(\boldsymbol{p})\boldsymbol{\Sigma}_k \bigr).

Cramer–Rao gives Var(z^)⪰Iz−1\mathrm{Var}(\hat{\boldsymbol{z}})\succeq I_{\boldsymbol{z}}^{-1} in ILR coordinates, with Iz=Jψ⊤I(p)JψI_{\boldsymbol{z}} =\mathbf{J}_{\boldsymbol{\psi}}^{\top} I(\boldsymbol{p}) \mathbf{J}_{\boldsymbol{\psi}} and Jψ=S(p)V\mathbf{J}_{\boldsymbol{\psi}} =\mathbf{S}(\boldsymbol{p})\mathbf{V} (jacobian_isometric_logistic()). The delta method maps the bound back to the simplex:

Var(p^)=JψIz−1JψT.\mathrm{Var}(\hat{\boldsymbol{p}}) = \mathbf{J}_{\boldsymbol{\psi}} I_{\boldsymbol{z}}^{-1} \mathbf{J}_{\boldsymbol{\psi}}^{\mathsf{T}}.

Value

An object of class decovart_fit.

References

Newman A, Liu C, Green M, Gentles A, Feng W, Xu Y, Hoang C, Diehn M, Alizadeh A (2015). “Robust Enumeration of Cell Subsets from Tissue Expression Profiles.” Nature methods, 12. doi:10.1038/nmeth.3337.

See Also

deconvolute_ratios(), deconvolute_ratios_Marquardt_Levenberg(), expected_fisher_unconstrained(), vcov_ilr_delta(), coef.decovart_fit(), vcov.decovart_fit(), confint.decovart_fit(), confint_decovart()

Examples

toy <- readRDS(system.file(
  "extdata", "toy_deconvolution.rds",
  package = "DeCovarT"
))
fit <- fit_decovart(
  signature_matrix = toy$signature_matrix,
  bulk_expression = toy$bulk_expression,
  Sigma = toy$Sigma,
  itmax = 40
)
coef(fit)
nobs(fit)

Firth-penalised DeCovarT fit

Description

Maximises ℓ(ρ)+12log⁡det⁡Iρ\ell(\boldsymbol{\rho})+\tfrac12\log\det I_{\boldsymbol{\rho}} on the ILR chart. The penalty is Jeffreys' prior for the expected Fisher information of one bulk column; extra replicate columns add only a constant, so they do not change the penalty gradient. The returned Wald covariance is that information multiplied by the number of columns and evaluated at the penalised estimate.

The penalty is differentiated numerically. The log-likelihood score stays analytic. This is not an exact removal of the O(N−1)O(N^{-1}) bias: the convolution is a curved exponential family.

Usage

fit_firth_decovart(
  y,
  mean_signature_matrix,
  Sigma,
  p_start = NULL,
  epsilon = 1e-08,
  itmax = 200L
)

Arguments

y

Numeric vector of length GG, or G×NG\times N matrix of replicate bulk profiles that share one composition; their log-likelihoods are summed (the pooled, population-level fit).

mean_signature_matrix

Numeric matrix μ∈MG×J\boldsymbol{\mu}\in\mathcal{M}_{G\times J} (plug-in means).

Sigma

Array of cell-type covariances in MG×G×J\mathcal{M}_{G\times G\times J}.

p_start

Start on the simplex. NULL uses the unpenalised MLE, pulled into the open simplex when that MLE lies on a face.

epsilon, itmax

Relative convergence tolerance and iteration budget passed to stats::nlminb().

Value

A list with coefficients, rho, loglik (unpenalised), penalty, penalised_loglik, vcov, vcov_rho and convergence.

References

Firth D (1993). “Bias Reduction of Maximum Likelihood Estimates.” Biometrika, 80(1), 27–38. doi:10.1093/biomet/80.1.27.

See Also

confint_decovart()

Other decovart_inference: bootstrap_decovart(), bootstrap_single_cell(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_decovart(), confint_ilr_backtransform(), confint_profile_decovart(), confint_profile_single_cell(), confint_single_cell(), equivariance_check_decovart(), fit_firth_single_cell(), lrt_decovart(), lrt_single_cell(), multistart_decovart(), reference_bootstrap_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

mu <- matrix(c(20, 40, 15, 40, 20, 25), nrow = 3)
colnames(mu) <- paste0("ct", 1:2)
Sigma <- array(c(diag(3), diag(3)), dim = c(3, 3, 2))
y <- drop(mu %*% c(0.6, 0.4))
fit_firth_decovart(y, mu, Sigma)$coefficients

Firth fit of one single-cell bulk sample

Description

Maximises the profile log-likelihood plus half the log determinant of the observed profile information. The penalty uses the negative profile Hessian, not an expected Fisher matrix. Scales stay concentrated out, so the penalty does not add a third free scale.

Usage

fit_firth_single_cell(
  y,
  mean_signature_matrix,
  Sigma,
  epsilon = 1e-08,
  itmax = 80L
)

Arguments

y

Bulk vector of length GG.

mean_signature_matrix

Reference mean, G×JG\times J.

Sigma

Cell-level covariances, G×G×JG\times G\times J.

epsilon

Relative function and step tolerances passed to stats::nlminb().

itmax

Maximum number of Newton iterations.

Value

A list with coefficients, rho, loglik, penalty, penalised_loglik, mean_scale, variance_scale and convergence.

See Also

confint_single_cell(), fit_firth_decovart()

Other decovart_inference: bootstrap_decovart(), bootstrap_single_cell(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_decovart(), confint_ilr_backtransform(), confint_profile_decovart(), confint_profile_single_cell(), confint_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), lrt_decovart(), lrt_single_cell(), multistart_decovart(), reference_bootstrap_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

mu <- matrix(c(2, 0.2, 0.2, 2, 0.2, 3, 0.2, 3), nrow = 4)
colnames(mu) <- c("ct1", "ct2")
sigma <- array(rep(diag(0.05, 4), 2), c(4, 4, 2))
y <- 20 * drop(mu %*% c(0.7, 0.3)) + c(0.3, -0.2, 0.1, -0.1)
fit_firth_single_cell(y, mu, sigma)$coefficients

Generate mean profiles from a target Gram matrix

Description

Builds μ∈MG×J\boldsymbol{\mu}\in\mathcal{M}_{G\times J} whose unit columns realise a prescribed Gram matrix RR (pairwise cosines) via the symmetric square root,

μ=s QR1/2,QTQ=IJ.\boldsymbol{\mu} = s\, \boldsymbol{Q} R^{1/2}, \qquad \boldsymbol{Q}^{\mathsf{T}}\boldsymbol{Q}=\boldsymbol{I}_{J}.

Then μTμ=s2R\boldsymbol{\mu}^{\mathsf{T}}\boldsymbol{\mu}=s^{2}R whenever RR has unit diagonal. The default RR is the equicorrelation (equicorrelation) matrix (1−ρ)I+ρ11T(1-\rho)I+\rho\mathbf{1}\mathbf{1}^{\mathsf{T}}, positive semidefinite for ρ∈[−1/(J−1),1]\rho\in[-1/(J-1),1]. Supply target_gram to set unequal pairwise cosines (for example two close pairs and a distant background), provided \(R\) is symmetric nonnegative-definite.

Usage

generate_mean_signature_matrix(
  n_genes,
  n_celltypes,
  mean_scale = 10,
  target_cosine = 0,
  target_gram = NULL,
  seed = NULL,
  nonnegative = FALSE,
  gene_names = NULL,
  celltype_names = NULL
)

Arguments

n_genes

Integer GG; must be at least n_celltypes.

n_celltypes

Integer J≥2J\ge 2.

mean_scale

Positive scalar ss (centroid norms). Default 10. Hold fixed when studying cosine / collinearity alone.

target_cosine

Numeric pairwise cosine ρ\rho for the equicorrelation Gram. Ignored when target_gram is supplied. Must lie in [−1/(J−1),1][-1/(J-1),1].

target_gram

Optional symmetric J×JJ\times J cosine matrix (unit diagonal). Overrides target_cosine.

seed

Optional integer. When supplied, the orthonormal frame Q\boldsymbol{Q} is drawn from a Gaussian QR; otherwise it is deterministic. Ignored when nonnegative = TRUE.

nonnegative

If TRUE, use a disjoint-support nonnegative frame so that μ≥0\boldsymbol{\mu}\ge 0 whenever R1/2≥0R^{1/2}\ge 0 (required by deconvolute_ratios()). The Gram is unchanged.

gene_names

Optional character vector of length GG.

celltype_names

Optional character vector of length JJ.

Details

Realisations are unique up to an orthogonal transformation of the gene space: replacing Q\boldsymbol{Q} by QU\boldsymbol{Q}U with UTU=IJU^{\mathsf{T}}U=\boldsymbol{I}_{J} leaves the Gram unchanged. By default Q\boldsymbol{Q} is a deterministic thin QR frame; pass seed for a Haar-like Gaussian frame (the global random state is restored). Cholesky R=LLTR=LL^{\mathsf{T}} is an alternative square root used in Monte Carlo simulation; see the how-to-build-synthetic-scenarios vignette appendix.

Value

Numeric matrix μ\boldsymbol{\mu} with dimensions G×JG\times J.

Examples

generate_mean_signature_matrix(
  n_genes = 6L,
  n_celltypes = 2L,
  target_cosine = 0.5
)
k <- matrix(c(1, 0.98, 0.2, 0.98, 1, 0.2, 0.2, 0.2, 1), 3, 3)
generate_mean_signature_matrix(
  n_genes = 8L,
  n_celltypes = 3L,
  target_gram = k
)

Helmert contrast matrix for isometric log-ratio coordinates

Description

Returns the J×(J−1)J\times(J-1) Helmert sub-matrix V\mathbf{V} with

VTV=IJ−1,VT1=0.\mathbf{V}^{\mathsf{T}}\mathbf{V}=\mathbf{I}_{J-1},\qquad \mathbf{V}^{\mathsf{T}}\mathbf{1}=\mathbf{0}.

Column kk contrasts the first kk parts against part k+1k+1. This is the standard ILR basis used with isometric_log_ratio() / isometric_logistic(); any other valid ILR basis is VQ\mathbf{V}Q for an orthogonal QQ, which rotates coordinates but leaves simplex geometry, eigenvalues of quadratic forms, and Var(p^)\mathrm{Var}(\hat{\boldsymbol{p}}) unchanged.

Usage

helmert_basis(n_parts)

Arguments

n_parts

Integer J≥2J\ge 2 (number of simplex parts).

Value

Numeric matrix J×(J−1)J\times(J-1).

Examples

V <- helmert_basis(3L)
crossprod(V)
drop(crossprod(V, rep(1, 3)))

Hessian tensor of the isometric logistic map

Description

Slice ii is

Hψi=pi(qiqiT−CV),\mathbf{H}_{\psi_i} = p_i\bigl(\boldsymbol{q}_i\boldsymbol{q}_i^{\mathsf{T}} -\mathbf{C}_V\bigr),

where vˉ=VTp\bar{\boldsymbol{v}}=\mathbf{V}^{\mathsf{T}}\boldsymbol{p}, qi=Vi⋅−vˉ\boldsymbol{q}_i=\mathbf{V}_{i\cdot}-\bar{\boldsymbol{v}} and CV=VTS(p)V\mathbf{C}_V=\mathbf{V}^{\mathsf{T}}\mathbf{S}(\boldsymbol{p}) \mathbf{V}. Array shape (J−1)×(J−1)×J(J-1)\times(J-1)\times J.

Usage

hessian_isometric_logistic(z, V = NULL)

Arguments

z

Numeric vector z∈RJ−1\boldsymbol{z}\in\mathbb{R}^{J-1}.

V

Optional ILR basis; see isometric_logistic().

Value

Numeric array used in the constrained Hessian chain rule.

Examples

hessian_isometric_logistic(c(0.2, -0.5))

Isometric logistic transform (ILR coordinates to the simplex)

Description

Inverse isometric log-ratio map ψ:z↦p\boldsymbol{\psi}:\boldsymbol{z}\mapsto\boldsymbol{p} with a Helmert basis V\mathbf{V},

p=softmax⁡(Vz)=C(exp⁡(Vz)).\boldsymbol{p} = \operatorname{softmax}(\mathbf{V}\boldsymbol{z}) = \mathcal{C}\bigl(\exp(\mathbf{V}\boldsymbol{z})\bigr).

No cell type is pinned as a reference. Evaluated with a log-sum-exp shift so large ∥z∥\|\boldsymbol{z}\| cannot overflow.

z=VTlog⁡p=VTclr⁡(p),\boldsymbol{z} = \mathbf{V}^{\mathsf{T}}\log\boldsymbol{p} = \mathbf{V}^{\mathsf{T}}\operatorname{clr}(\boldsymbol{p}),

which is well-defined because VT1=0\mathbf{V}^{\mathsf{T}}\mathbf{1}=0. Requires a strictly positive composition (open simplex).

Usage

isometric_logistic(z, V = NULL)

isometric_log_ratio(p, V = NULL)

Arguments

z

Numeric vector z∈RJ−1\boldsymbol{z}\in\mathbb{R}^{J-1}.

V

Optional ILR basis; see isometric_logistic().

p

Numeric vector on the open simplex.

Value

Numeric vector p\boldsymbol{p} on the unit simplex.

Numeric vector z∈RJ−1\boldsymbol{z}\in\mathbb{R}^{J-1}.

See Also

isometric_log_ratio(), helmert_basis()

Examples

z <- c(0.2, -0.5)
p <- isometric_logistic(z)
sum(p)
isometric_log_ratio(p)
isometric_logistic(c(800, 1200))

Jacobian of the isometric logistic map

Description

Jψ(z)=∂p∂zT=S(p)V,\mathbf{J}_{\boldsymbol{\psi}}(\boldsymbol{z}) = \frac{\partial\boldsymbol{p}}{\partial\boldsymbol{z}^{\mathsf{T}}} = \mathbf{S}(\boldsymbol{p})\mathbf{V},

with S(p)=diag⁡(p)−ppT\mathbf{S}(\boldsymbol{p})=\operatorname{diag}(\boldsymbol{p}) -\boldsymbol{p}\boldsymbol{p}^{\mathsf{T}}.

Usage

jacobian_isometric_logistic(z, V = NULL)

Arguments

z

Numeric vector z∈RJ−1\boldsymbol{z}\in\mathbb{R}^{J-1}.

V

Optional ILR basis; see isometric_logistic().

Value

Numeric matrix J×(J−1)J\times(J-1).

Examples

jacobian_isometric_logistic(c(0.2, -0.5))

Likelihood-ratio test for cellular ratios

Description

Tests H0:pj=cj, j∈AH_0:p_j=c_j,\,j\in A against the unrestricted simplex with the profile likelihood-ratio statistic

D=2{ℓ(p^)−ℓ(p^(0))},D = 2\bigl\{ \ell(\hat{\boldsymbol{p}}) -\ell(\hat{\boldsymbol{p}}^{(0)}) \bigr\},

where p^(0)\hat{\boldsymbol{p}}^{(0)} is the restricted MLE of restricted_mle_decovart(). Unlike the Wald interval of confint.decovart_fit(), DD is invariant under the additive log-ratio reparametrisation, because a smooth bijection leaves likelihood ratios unchanged.

Usage

lrt_decovart(
  y,
  mean_signature_matrix,
  Sigma,
  null_value,
  n_boundary = NULL,
  boundary_tol = 1e-06,
  epsilon = 10^-8,
  itmax = 500L
)

Arguments

y

Numeric vector of length GG, or G×NG\times N matrix of replicate bulk profiles that share one composition; their log-likelihoods are summed (the pooled, population-level fit).

mean_signature_matrix

Numeric matrix μ∈MG×J\boldsymbol{\mu}\in\mathcal{M}_{G\times J} (plug-in means).

Sigma

Array of cell-type covariances in MG×G×J\mathcal{M}_{G\times G\times J}.

null_value

Numeric vector of hypothesised ratios named by cell types, e.g. c(celltype_2 = 0).

n_boundary

Optional integer override for the number of active boundary constraints; NULL infers it from null_value, counting the zeros implied when the fixed values sum to one.

boundary_tol

Distance from 0 or 1 below which a null value counts as being on the boundary.

epsilon, itmax

Relative convergence tolerance and iteration budget passed to stats::nlminb().

Details

Interior null. If every tested ratio lies strictly inside (0,1)(0,1), DD is asymptotically χ∣A∣2\chi^{2}_{|A|} (Wilks 1938).

Boundary null. A null such as pj=0p_j=0 lets the parameter move in one direction only, so the local parameter space is a half-line rather than a vector space and Wilks' theorem fails. The limit is the chi-bar-square mixture of chi_bar_square_pvalue(): for a single active constraint, D⇝12χ02+12χ12D\rightsquigarrow\tfrac{1}{2}\chi^{2}_{0}+\tfrac{1}{2}\chi^{2}_{1} Chernoff (1954), generalised to several active constraints by Self and Liang (1987). Boundary calibration is selected automatically when any tested value is within boundary_tol of 0, or when the tested values sum to one, which forces every other ratio to zero (a null pj=1p_j=1 fixes J−1J-1 coordinates on faces); override with n_boundary. The binomial weights of chi_bar_square_pvalue() are exact for orthogonal constraints; for several correlated faces they approximate the weights of the tangent cone, and bootstrap_decovart() with null_value gives a calibration that does not rely on them.

Replication. Both calibrations are population statements: they need replicate bulk samples that genuinely share one composition p\boldsymbol{p}. A single GG-vector supplies no replication (genes are dependent through Σ(p)\boldsymbol{\Sigma}(\boldsymbol{p}), and adding bulk samples with distinct compositions adds parameters as fast as observations). Pass several replicate columns in y, or calibrate by bootstrap_decovart().

Value

A one-row data frame with columns statistic, n_boundary, df_interior, p_value, loglik_full, loglik_null, and calibration.

References

Wilks SS (1938). “The Large-Sample Distribution of the Likelihood Ratio for Testing Composite Hypotheses.” The Annals of Mathematical Statistics, 9(1), 60–62. ISSN 0003-4851. doi:10.1214/aoms/1177732360.

Self SG, Liang K (1987). “Asymptotic Properties of Maximum Likelihood Estimators and Likelihood Ratio Tests under Nonstandard Conditions.” Journal of the American Statistical Association, 82(398), 605–610. ISSN 0162-1459. doi:10.1080/01621459.1987.10478472.

See Also

chi_bar_square_pvalue(), confint_profile_decovart(), bootstrap_decovart()

Other decovart_inference: bootstrap_decovart(), bootstrap_single_cell(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_decovart(), confint_ilr_backtransform(), confint_profile_decovart(), confint_profile_single_cell(), confint_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), fit_firth_single_cell(), lrt_single_cell(), multistart_decovart(), reference_bootstrap_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

mu <- matrix(c(20, 22, 18, 22, 20, 24), nrow = 2)
colnames(mu) <- paste0("ct", 1:3)
Sigma <- array(c(diag(2), diag(2), diag(2)), dim = c(2, 2, 3))
y <- drop(mu %*% c(0.5, 0.5, 0))
lrt_decovart(y, mu, Sigma, null_value = c(ct3 = 0))

Likelihood-ratio test for one single-cell composition

Description

Compares the profiled maximum with the maximum on fixed (passed as null_value). An interior null uses Wilks' chi-square. A null on a face uses the chi-bar-square mixture of chi_bar_square_pvalue(). The two technical scales are profiled out under both hypotheses, so the degrees of freedom count composition constraints only.

Usage

lrt_single_cell(
  y,
  mean_signature_matrix,
  Sigma,
  null_value,
  n_boundary = NULL,
  boundary_tol = 1e-06,
  epsilon = 1e-08,
  itmax = 100L
)

Arguments

y

Bulk vector of length GG.

mean_signature_matrix

Reference mean, G×JG\times J.

Sigma

Cell-level covariances, G×G×JG\times G\times J.

null_value

Named hypothesised ratios.

n_boundary, boundary_tol

As in lrt_decovart().

epsilon

Relative function and step tolerances passed to stats::nlminb().

itmax

Maximum number of Newton iterations.

Value

A one-row data frame, as for lrt_decovart().

See Also

confint_profile_single_cell(), lrt_decovart()

Other decovart_inference: bootstrap_decovart(), bootstrap_single_cell(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_decovart(), confint_ilr_backtransform(), confint_profile_decovart(), confint_profile_single_cell(), confint_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), fit_firth_single_cell(), lrt_decovart(), multistart_decovart(), reference_bootstrap_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

mu <- matrix(c(2, 0.2, 0.2, 2, 0.2, 3, 0.2, 3), nrow = 4)
colnames(mu) <- c("ct1", "ct2")
sigma <- array(rep(diag(0.05, 4), 2), c(4, 4, 2))
y <- 20 * drop(mu %*% c(1, 0)) + c(0.2, -0.1, 0.05, 0.05)
lrt_single_cell(y, mu, sigma, null_value = c(ct2 = 0))

Refit DeCovarT from several random starts to probe multimodality

Description

The realised DeCovarT log-likelihood is not globally concave, so a successful convergence code, a small relative-distance-to-minimum and a negative-definite local Hessian together establish a local maximum only. Refitting from independent Dirichlet(1,…,1)(1,\ldots,1) starts and comparing the attained log-likelihoods is the cheapest direct probe of a second, equally good or better mode.

Usage

multistart_decovart(
  y,
  mean_signature_matrix,
  Sigma,
  n_starts = 5L,
  solver = deconvolute_ratios_Newton_Raphson,
  loglik_tol = 1e-04,
  dirichlet_alpha = 1,
  ...
)

Arguments

y

Numeric vector (or one-column matrix) y∈RG\boldsymbol{y}\in\mathbb{R}^{G}.

mean_signature_matrix

Numeric matrix μ∈MG×J\boldsymbol{\mu}\in\mathcal{M}_{G\times J} (plug-in means).

Sigma

Array of cell-type covariances in MG×G×J\mathcal{M}_{G\times G\times J}.

n_starts

Number of Dirichlet restarts (in addition to the equi-balanced start).

solver

Solver accepting initial_p and return_model; defaults to deconvolute_ratios_Newton_Raphson().

loglik_tol

Two starts count as reaching the same mode when their log-likelihoods differ by less than this amount.

dirichlet_alpha

Concentration for those random starts (default 1). See starting_simplex().

...

Passed to solver (e.g. epsilon, itmax).

Value

A list with coefficients (best fit), loglik, loglik_range, starts (matrix of per-start estimates), logliks, and multimodal.

See Also

boundary_diagnostics(), fit_decovart()

Other decovart_inference: bootstrap_decovart(), bootstrap_single_cell(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_decovart(), confint_ilr_backtransform(), confint_profile_decovart(), confint_profile_single_cell(), confint_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), fit_firth_single_cell(), lrt_decovart(), lrt_single_cell(), reference_bootstrap_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

mu <- matrix(c(20, 40, 15, 40, 20, 25), nrow = 3)
colnames(mu) <- paste0("ct", 1:2)
Sigma <- array(c(diag(3), diag(3)), dim = c(3, 3, 2))
y <- drop(mu %*% c(0.6, 0.4))
set.seed(1)
multistart_decovart(y, mu, Sigma, n_starts = 3L)$multimodal

MixSim-style pairwise overlap via stratified Sobol Monte Carlo

Description

Estimates the MixSim Omega map Ωjℓ=Pr⁡X∼fj(πℓfℓ(X)>πjfj(X))\Omega_{j\ell}=\Pr_{X\sim f_j}(\pi_\ell f_\ell(X)>\pi_j f_j(X)) by drawing n_mc quasi-Monte Carlo points from each component (inverse-transform sampling of a Sobol sequence through the Cholesky factor) and comparing log-densities. The average pairwise overlap BarOmega is the unweighted mean of Ωjℓ+Ωℓj\Omega_{j\ell}+\Omega_{\ell j} over j<ℓj<\ell, matching MixSim::overlap() / FSDA MixSim.

Usage

overlap_gaussian_mc(true_theta, n_mc = 10000L, seed = NULL, J = NULL)

Arguments

true_theta

List with p (length JJ), mu (G×JG\times J) and sigma (G×G×JG\times G\times J).

n_mc

Draws per component (default 10000).

seed

Optional RNG seed (passed to Sobol randomisation and to the uniform fallback).

J

Optional number of components.

Details

For two equal-weight components this pairwise overlap equals the histogram similarity ∫min⁡(fj,fℓ)\int\min(f_j,f_\ell) and therefore 1−TV(fj,fℓ)1-\mathrm{TV}(f_j,f_\ell) (Nielsen and Sun 2018).

Value

A list with BarOmega, MaxOmega, and OmegaMap.

References

Nielsen F, Sun K (2018). “Guaranteed Deterministic Bounds on the Total Variation Distance between Univariate Mixtures.” arXiv:1806.11311. doi:10.48550/arxiv.1806.11311. https://arxiv.org/abs/1806.11311.

See Also

compute_average_overlap(), compute_average_riemannian()

Examples

set.seed(1)
theta <- list(
  p = c(0.5, 0.5),
  mu = cbind(c(0, 0), c(3, 0)),
  sigma = array(c(diag(2), diag(2)), dim = c(2, 2, 2))
)
overlap_gaussian_mc(theta, n_mc = 2000L)$BarOmega

Pivot Monte Carlo proportion estimates to a long table

Description

Turns the optimisation block of run_simulation_benchmark() into one row per replicate, algorithm, and cell type, with the matching true proportion from theta_true. The raincloud intervals drawn from this table are empirical Monte Carlo quantiles of p^\hat{\boldsymbol{p}}, not confidence intervals for p\boldsymbol{p} (Allen et al. 2019).

Usage

pivot_mc_estimates(benchmark)

Arguments

benchmark

List returned by run_simulation_benchmark().

Value

A tibble with algorithm, cell_type, estimate, p_true, error, sample_id, and any scenario metadata columns.

References

Allen M, Poggiali D, Whitaker K, Marshall TR, van Langen J, Kievit RA (2019). “Raincloud Plots: A Multi-Platform Tool for Robust Data Visualization.” Wellcome Open Research, 4, 63. ISSN 2398-502X. doi:10.12688/wellcomeopenres.15191.2. https://wellcomeopenresearch.org/articles/4-63.

See Also

plot_mc_raincloud(), plot_mc_forest()

Examples

set.seed(1)
genes <- paste0("g", 1:2)
cts <- paste0("ct", 1:2)
mu <- matrix(c(20, 22, 22, 20), nrow = 2, dimnames = list(genes, cts))
Sigma <- array(
  c(1, 0, 0, 1, 1, 0, 0, 1),
  dim = c(2, 2, 2),
  dimnames = list(genes, genes, cts)
)
out <- run_simulation_benchmark(
  tibble::tibble(
    ID = "B1",
    true_theta = list(list(p = c(0.5, 0.5), mu = mu, sigma = Sigma))
  ),
  deconvolution_functions = list(
    "nnls" = list(FUN = deconvolute_ratios_nnls)
  ),
  n = 2,
  cores = 1
)
nrow(pivot_mc_estimates(out))

Tile heatmap of algorithm-similarity correlations

Description

ggplot2::geom_tile() display of algorithm_similarity(), with rows and columns ordered by Ward D2 clustering of \(1-r\). Optional dendrogram via ggdendro (Suggests), drawn to the right of the tiles with leaves flush against the heatmap and Ward merge heights on the axis.

Usage

plot_algorithm_similarity(
  benchmark,
  facet_rows = NULL,
  facet_cols = NULL,
  dendrogram = FALSE
)

Arguments

benchmark

List from run_simulation_benchmark().

facet_rows, facet_cols

Optional scenario columns. When supplied, correlations are computed inside each facet cell.

dendrogram

If TRUE, attach a ggdendro ggplot to the right of the tiles as attribute "dendrogram" (ignored when scenario facets are used).

Value

A ggplot object.

See Also

algorithm_similarity(), plot_mc_metric_dots()

Examples

set.seed(1)
genes <- paste0("g", 1:2)
cts <- paste0("ct", 1:2)
mu <- matrix(c(20, 22, 22, 20), nrow = 2, dimnames = list(genes, cts))
Sigma <- array(
  c(1, 0, 0, 1, 1, 0, 0, 1),
  dim = c(2, 2, 2),
  dimnames = list(genes, genes, cts)
)
out <- run_simulation_benchmark(
  tibble::tibble(
    true_theta = list(list(p = c(0.5, 0.5), mu = mu, sigma = Sigma))
  ),
  deconvolution_functions = list(
    "nnls" = list(FUN = deconvolute_ratios_nnls),
    "rlm" = list(FUN = deconvolute_ratios_rlm)
  ),
  n = 4,
  cores = 1
)
if (requireNamespace("ggplot2", quietly = TRUE)) {
  plot_algorithm_similarity(out)
}

Forest plot of ADEMP Monte Carlo summaries

Description

Dot-and-whisker display of bias, RMSE, MAE, coverage, mean interval width, SE/SD ratio, and optimiser failure rate by algorithm and cell type (Allen et al. 2019). Coverage whiskers are the Wilson interval already stored on monte_carlo (coverage_mc_interval(); (Wilson 1927)): they are intervals for the coverage rate, not for pjp_j. Bias is referenced at 0; coverage at 0.95; SE/SD (se_sd_ratio, mean model SE / empirical SD) at 1. Interior Wald SEs come from vcov_ilr_delta(). Pairwise algorithm contrasts (MAE differences versus a reference solver on the same Monte Carlo replicates) can be read from the raincloud of paired errors; they do not need a second bootstrap.

Usage

plot_mc_forest(
  benchmark,
  facet_rows = NULL,
  facet_cols = NULL,
  metrics = c("bias", "rmse", "mae", "coverage", "mean_interval_width", "se_sd_ratio",
    "failure_rate")
)

Arguments

benchmark

List from run_simulation_benchmark(), or a table from pivot_mc_estimates().

facet_rows, facet_cols

Optional column names for ggplot2::facet_grid() rows and columns.

metrics

Character vector of summaries to display.

Value

A ggplot object.

References

Allen M, Poggiali D, Whitaker K, Marshall TR, van Langen J, Kievit RA (2019). “Raincloud Plots: A Multi-Platform Tool for Robust Data Visualization.” Wellcome Open Research, 4, 63. ISSN 2398-502X. doi:10.12688/wellcomeopenres.15191.2. https://wellcomeopenresearch.org/articles/4-63.

Wilson EB (1927). “Probable Inference, the Law of Succession, and Statistical Inference.” Journal of the American Statistical Association, 22(158), 209–212. ISSN 0162-1459. doi:10.2307/2276774.

See Also

plot_mc_raincloud(), coverage_mc_interval(), vcov_ilr_delta(), theme_decovart_facets()

Examples

set.seed(1)
genes <- paste0("g", 1:2)
cts <- paste0("ct", 1:2)
mu <- matrix(c(20, 22, 22, 20), nrow = 2, dimnames = list(genes, cts))
Sigma <- array(
  c(1, 0, 0, 1, 1, 0, 0, 1),
  dim = c(2, 2, 2),
  dimnames = list(genes, genes, cts)
)
out <- run_simulation_benchmark(
  tibble::tibble(
    n_genes = 2L,
    cosine = 0,
    true_theta = list(list(p = c(0.5, 0.5), mu = mu, sigma = Sigma))
  ),
  deconvolution_functions = list(
    "nnls" = list(FUN = deconvolute_ratios_nnls)
  ),
  n = 4,
  cores = 1
)
if (requireNamespace("ggplot2", quietly = TRUE)) {
  plot_mc_forest(out, facet_cols = "cosine")
}

Faceted dot plot of several ADEMP metrics

Description

One panel per metric; colour is a min-max normalised score with \(1\) = best inside that metric (and facet). Do not map a second primary score to point size. Coverage is scored as ∣C^−0.95∣|\widehat C-0.95|, so over-wide intervals that inflate coverage are not rewarded. There is no default weighted composite.

Usage

plot_mc_metric_dots(
  benchmark,
  facet_rows = NULL,
  facet_cols = NULL,
  metrics = c("rmse", "mae", "coverage", "mean_interval_width"),
  weights = NULL
)

Arguments

benchmark

List from run_simulation_benchmark(), or a table from pivot_mc_estimates().

facet_rows, facet_cols

Optional column names for ggplot2::facet_grid() rows and columns.

metrics

Character vector of summaries to display.

weights

Optional named numeric weights (sum to 1) used only to add a composite facet beside the components.

Value

A ggplot object.

See Also

plot_mc_forest(), plot_algorithm_similarity()

Examples

set.seed(1)
genes <- paste0("g", 1:2)
cts <- paste0("ct", 1:2)
mu <- matrix(c(20, 22, 22, 20), nrow = 2, dimnames = list(genes, cts))
Sigma <- array(
  c(1, 0, 0, 1, 1, 0, 0, 1),
  dim = c(2, 2, 2),
  dimnames = list(genes, genes, cts)
)
out <- run_simulation_benchmark(
  tibble::tibble(
    cosine = 0,
    true_theta = list(list(p = c(0.5, 0.5), mu = mu, sigma = Sigma))
  ),
  deconvolution_functions = list(
    "nnls" = list(FUN = deconvolute_ratios_nnls)
  ),
  n = 4,
  cores = 1
)
if (requireNamespace("ggplot2", quietly = TRUE)) {
  plot_mc_metric_dots(out, facet_cols = "cosine")
}

Horizontal raincloud of Monte Carlo proportion estimates

Description

Half-eye densities, dots, and empirical 50% / 95% intervals (Allen et al. 2019) for the Monte Carlo sampling distribution of p^j\hat p_j. Default aesthetics: numeric axis is the estimation error p^j−pj⋆\hat p_j-p_j^{\star} (so a vertical line at 0 is bias); the y-axis is the cell type; colour and dodge group the deconvolution algorithm. Optional facet_rows / facet_cols split scenarios (for example number of genes versus pairwise cosine).

Usage

plot_mc_raincloud(
  benchmark,
  quantity = c("error", "estimate"),
  facet_rows = NULL,
  facet_cols = NULL,
  .width = c(0.5, 0.95),
  include_dots = TRUE,
  max_dots = 2000L,
  max_rows = NULL,
  dodge_width = 0.95,
  slab_scale = 1.4,
  slab_alpha = 1,
  category_spacing = 1,
  slab = c("halfeye", "dotsinterval")
)

Arguments

benchmark

List from run_simulation_benchmark(), or a table from pivot_mc_estimates().

quantity

"error" (default) or "estimate".

facet_rows, facet_cols

Optional column names for ggplot2::facet_grid() rows and columns.

.width

Passed to ggdist::stat_halfeye(); default c(0.5, 0.95).

include_dots

If FALSE, omit ggdist::stat_dots() (use this on large Monte Carlo tables: the Fortran QP in nudge_bins cannot take long vectors).

max_dots

Maximum rows passed to stat_dots when include_dots = TRUE. Extra rows are subsampled.

max_rows

Optional cap on the plotting table (half-eye and dots). When the Monte Carlo stack is huge, subsample before drawing.

dodge_width

Width passed to ggplot2::position_dodge().

slab_scale

Slab height for ggdist::stat_halfeye().

slab_alpha

Transparency of the density slab (1 is opaque).

category_spacing

Vertical gap between discrete y categories (cell types). Values greater than 1 insert extra space so slabs from neighbouring types do not overlap; the axis still shows the original labels.

slab

"halfeye" (default KDE raincloud) or "dotsinterval" (quantile dots plus interval; avoids empty bounded slabs).

Details

When quantity = "estimate" and slab = "halfeye", slabs use a bounded kernel on [0,1][0,1] (ggdist::density_bounded()) with a coloured outline (slab_colour / slab_linewidth). Vertex pile-up can leave that KDE empty; slab = "dotsinterval" (ggdist::stat_dotsinterval()) draws quantile dots instead. True proportions are labelled at the top of each panel (the most abundant type to the right of its reference line; the others to the left).

The inner interval is the central 50% of Monte Carlo replicates; the outer interval is the central 95%. These are not confidence intervals for pjp_j.

Value

A ggplot object.

References

Allen M, Poggiali D, Whitaker K, Marshall TR, van Langen J, Kievit RA (2019). “Raincloud Plots: A Multi-Platform Tool for Robust Data Visualization.” Wellcome Open Research, 4, 63. ISSN 2398-502X. doi:10.12688/wellcomeopenres.15191.2. https://wellcomeopenresearch.org/articles/4-63.

See Also

plot_mc_forest(), pivot_mc_estimates()

Examples

set.seed(1)
genes <- paste0("g", 1:2)
cts <- paste0("ct", 1:2)
mu <- matrix(c(20, 22, 22, 20), nrow = 2, dimnames = list(genes, cts))
Sigma <- array(
  c(1, 0, 0, 1, 1, 0, 0, 1),
  dim = c(2, 2, 2),
  dimnames = list(genes, genes, cts)
)
out <- run_simulation_benchmark(
  tibble::tibble(
    n_genes = 2L,
    cosine = 0,
    true_theta = list(list(p = c(0.5, 0.5), mu = mu, sigma = Sigma))
  ),
  deconvolution_functions = list(
    "nnls" = list(FUN = deconvolute_ratios_nnls)
  ),
  n = 4,
  cores = 1
)
if (requireNamespace("ggdist", quietly = TRUE)) {
  plot_mc_raincloud(out, facet_cols = "cosine")
}

Reference-based bootstrap for signature and composition uncertainty

Description

The frequentist DeCovarT likelihood treats μ\boldsymbol{\mu} and each Σj\boldsymbol{\Sigma}_j as known. In a labelled, reference-based model those moments come from purified profiles whose cell-type names stay attached. This bootstrap resamples the experimental units that generated the reference (donors, by default), re-estimates the plug-in moments and refits (Efron 1979).

Usage

reference_bootstrap_decovart(
  y,
  reference_profiles,
  donor_ids = NULL,
  method = c("donors", "cells"),
  n_boot = 199L,
  level = 0.95,
  regenerate_bulk = FALSE,
  p = NULL,
  n_replicates = NULL,
  ridge = 1e-06,
  epsilon = 10^-8,
  itmax = 500L
)

Arguments

y

Numeric vector of length GG, or G×NG\times N matrix of replicate bulk profiles that share one composition; their log-likelihoods are summed (the pooled, population-level fit).

reference_profiles

Named list of G×njG\times n_j matrices, one purified sample matrix per cell type (at least two columns each). The list order fixes the column order of the signature.

donor_ids

Donor (or other clustering) labels: a list named like reference_profiles, with one label per purified column. Required for method = "donors".

method

One of "donors" (default) or "cells".

n_boot

Number of bootstrap replicates.

level

Confidence level for percentile intervals.

regenerate_bulk

If TRUE, simulate a new bulk from the fitted model and refit it with the bootstrapped moments, instead of keeping the observed Y\boldsymbol{Y} fixed.

p

Composition to simulate from. Defaults to the MLE computed from y (or the restricted MLE under null_value).

n_replicates

Number of bulk columns simulated per replicate; defaults to the number of observed columns.

ridge

Non-negative ridge multiple for the sample covariance of each cell type. The default 1e-6 is relative to the mean diagonal.

epsilon, itmax

Relative convergence tolerance and iteration budget passed to stats::nlminb().

Details

Permuting gene order, or permuting cell-type labels of an already averaged signature, is not a bootstrap: the first destroys the gene-wise pairing of bulk and reference, and the second is algebraic equivariance rather than a source of uncertainty (see equivariance_check_decovart() and vignette("theory-DeCovarT-MLE-properties")). The two method options instead resample, with replacement, units that actually vary.

donors

Cluster bootstrap: resample donor identifiers within each cell type, take every purified column of the sampled donors (a donor drawn twice is included twice), rebuild (μ,Σj)(\boldsymbol{\mu},\boldsymbol{\Sigma}_j), and refit. This is the default: it targets biological / reference-population uncertainty rather than technical cell-level noise alone.

cells

Resample purified columns independently within each cell type (with replacement) and rebuild the plug-in moments. Use this when donor labels are unavailable.

The observed bulk is kept fixed unless regenerate_bulk = TRUE, in which case each replicate also draws Y(b)\boldsymbol{Y}^{(b)} from the fitted model, at p (the observed MLE by default) and the moments of the full reference, and refits it with the bootstrapped moments S(b)S^{(b)}. Drawing Y(b)\boldsymbol{Y}^{(b)} from S(b)S^{(b)} instead would make each refit consistent with its own moments, so the reference error would not reach p^(b)\hat{\boldsymbol{p}}^{(b)}: with 60 cells per type, that variant covered a known composition 76\ level, against 92\

When a cell type has fewer purified columns than genes, the sample covariance is singular. A ridge ridge * mean(diag(S)) is then added on the diagonal so that Σ(p)\boldsymbol{\Sigma}(\boldsymbol{p}) remains factorisable. Set ridge = 0 to disable that guard (and accept possible Cholesky failures).

Value

A list with method, estimates (J×J\times n_boot), interval (percentile bounds), n_failed, mean_signature_matrix / Sigma from the original reference, and p_hat from the original plug-in fit.

References

Efron B (1979). “Bootstrap Methods: Another Look at the Jackknife.” The Annals of Statistics, 7(1), 1–26. doi:10.1214/aos/1176344552.

See Also

bootstrap_decovart(), equivariance_check_decovart()

Other decovart_inference: bootstrap_decovart(), bootstrap_single_cell(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_decovart(), confint_ilr_backtransform(), confint_profile_decovart(), confint_profile_single_cell(), confint_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), fit_firth_single_cell(), lrt_decovart(), lrt_single_cell(), multistart_decovart(), reference_bootstrap_single_cell(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

set.seed(1)
mu <- matrix(c(20, 40, 15, 40, 20, 25), nrow = 3)
colnames(mu) <- paste0("ct", 1:2)
Sigma <- array(c(diag(3), diag(3)), dim = c(3, 3, 2))
refs <- lapply(seq_len(2), function(j) {
  t(MASS::mvrnorm(n = 8, mu = mu[, j], Sigma = Sigma[, , j]))
})
names(refs) <- colnames(mu)
donor_ids <- lapply(refs, function(x) {
  rep(paste0("d", 1:2), length.out = ncol(x))
})
y <- drop(mu %*% c(0.6, 0.4))
reference_bootstrap_decovart(
  y,
  refs,
  donor_ids = donor_ids,
  n_boot = 15
)$interval

Reference bootstrap for the single-cell mixture

Description

Resamples the reference that produced (μ~,Σj)(\tilde{\boldsymbol{\mu}},\boldsymbol{\Sigma}_j) and refits deconvolute_single_cell(). resample = NULL uses donors when every cell type has at least two donor labels, and otherwise resamples columns within each cell type. The within-type resample regenerates the bulk from the fitted single-cell law, at the original (p^,c^,ϕ^)(\hat{\boldsymbol{p}},\hat c,\hat\phi) and the original moments, and refits it with the resampled moments, so that both the bulk noise and the reference error enter the interval.

Shuffling cell-type names is not a reference bootstrap. It does not redraw cells or donors, and the likelihood is equivariant under a relabeling, so the draw would not be a source of uncertainty. That check remains equivariance_check_decovart() for the population model.

Usage

reference_bootstrap_single_cell(
  y,
  reference_profiles,
  donor_ids = NULL,
  resample = NULL,
  n_boot = 100L,
  level = 0.95,
  regenerate_bulk = NULL,
  ridge = 1e-06,
  epsilon = 1e-06,
  itmax = 40L
)

Arguments

y

Bulk vector of length GG.

reference_profiles

Named list of gene-by-cell (or gene-by-replicate) matrices, one per cell type.

donor_ids

Optional list of donor labels, aligned with reference_profiles.

resample

"donors", "cells", or NULL for the automatic rule above.

n_boot

Number of replicates.

level

Confidence level.

regenerate_bulk

NULL regenerates the bulk only for the within-type resample. TRUE or FALSE overrides that rule.

ridge

Ridge passed to the sample covariance, relative to its mean diagonal.

epsilon

Relative function and step tolerances passed to stats::nlminb().

itmax

Maximum number of Newton iterations.

Value

A list with resample, regenerate_bulk, estimates, interval, n_failed, scales, scale_interval and fit.

See Also

reference_bootstrap_decovart(), confint_single_cell()

Other decovart_inference: bootstrap_decovart(), bootstrap_single_cell(), boundary_diagnostics(), chi_bar_square_pvalue(), confint_decovart(), confint_ilr_backtransform(), confint_profile_decovart(), confint_profile_single_cell(), confint_single_cell(), equivariance_check_decovart(), fit_firth_decovart(), fit_firth_single_cell(), lrt_decovart(), lrt_single_cell(), multistart_decovart(), reference_bootstrap_decovart(), restricted_mle_decovart(), restricted_mle_single_cell()

Examples

set.seed(1)
mu <- cbind(ct1 = c(2, 2, 0.2, 0.2), ct2 = c(0.2, 0.2, 3, 3))
sigma <- array(rep(diag(0.05, 4), 2), c(4, 4, 2))
refs <- lapply(seq_len(2), function(j) {
  t(MASS::mvrnorm(n = 6, mu = mu[, j], Sigma = sigma[, , j]))
})
names(refs) <- colnames(mu)
y <- 20 * drop(mu %*% c(0.7, 0.3)) + rnorm(4, sd = 0.2)
reference_bootstrap_single_cell(y, refs, n_boot = 4)$resample

Repair a numeric vector onto the unit simplex

Description

Clips numerical under/overflow and renormalises so that 1Tp=1\mathbf{1}^{\mathsf{T}}\boldsymbol{p}=1 and p≥0\boldsymbol{p}\ge\mathbf{0}. This is a repair / renormalisation step for estimated proportions, not a Euclidean projection onto the simplex and not a statistical-identifiability constraint.

When open = TRUE, entries on the closed-simplex boundary are nudged into the relative interior so that isometric_log_ratio() is defined (ILR / Marquardt / Newton starts). That open-simplex path is the former role of starting_simplex().

Usage

repair_simplex(
  p,
  tolerance = 100 * .Machine$double.eps,
  open = FALSE,
  nms = NULL
)

Arguments

p

Numeric vector p∈RJ\boldsymbol{p}\in\mathbb{R}^{J}.

tolerance

Non-negative tolerance for treating entries as zero (default 100 * .Machine$double.eps).

open

Logical; if TRUE, push the repaired vector off the boundary into the open simplex.

nms

Optional names attached to the returned vector (e.g. cell-type colnames).

Value

Numeric vector on the simplex ΔJ−1\Delta^{J-1} (open when open = TRUE).

See Also

compositions::clo() for compositional closure.

Examples

repair_simplex(c(0.2, 0.3, 0.5 + 1e-12))
repair_simplex(c(1, 0, 0), open = TRUE)

Cell-size conventions for a single-cell reference

Description

Turns nonnegative gene-by-cell-type moments into the reference used by the single-cell mixture. Column jj of the supplied mean is normalised to a composition θ⋅j\boldsymbol{\theta}_{\cdot j}. Two conventions are available; the default is the raw-covariance, cell-fraction model of the article.

  • cellular_raw (default). Mean μ~⋅j=Sjθ⋅j\tilde{\boldsymbol{\mu}}_{\cdot j}=S_j\boldsymbol{\theta}_{\cdot j} with the covariance left as supplied. Keeping SjS_j in the mean makes the fitted composition a cell fraction, not a transcript fraction.

  • transcript_raw. Library-normalised mean θ⋅j\boldsymbol{\theta}_{\cdot j} and per-transcript covariance Σj/Sj\boldsymbol{\Sigma}_j/S_j. The fitted composition is the transcript fraction q\boldsymbol{q}. The physical generator is still the cellular model: generative_mean and generative_sigma are the moments simulate_bulk_from_cells() should draw from.

Usage

rescale_single_cell_moments(
  moments,
  cell_size,
  reference = c("cellular_raw", "transcript_raw")
)

Arguments

moments

List with mean_profiles (G×JG\times J) and covariance_matrices (G×G×JG\times G\times J), as returned by simulate_hierarchical_grn_moments() with nonnegative = TRUE.

cell_size

Positive cell sizes SjS_j. A scalar is repeated once per cell type. For two types the scenarios use (1,1)(1, 1) and (1,5)(1, 5); (1,2,5)(1, 2, 5) is the same idea for three types.

reference

cellular_raw or transcript_raw.

Value

A list with the matrices passed to the estimator (mean_signature_matrix, Sigma), the cellular generator (generative_mean, generative_sigma), cell_size, composition (θ\boldsymbol{\theta}), reference and estimand ("cellular" or "transcript").

See Also

simulate_hierarchical_grn_moments(), simulate_bulk_from_cells(), single_cell_scenario_design()

Examples

mu <- cbind(c(2, 2, 0, 0), c(0, 0, 1, 1))
sigma <- array(rep(diag(4), 2), dim = c(4, 4, 2))
moments <- list(mean_profiles = mu, covariance_matrices = sigma)
ref <- rescale_single_cell_moments(moments, c(1, 5))
colSums(ref$generative_mean)

Simulate bulk mixtures and benchmark deconvolution algorithms

Description

Wrapper around simulate_bulk_mixture(), deconvolute_ratios(), and compute_benchmark_metrics() (called inside deconvolute_ratios()). Each row of scenario_config describes one generative model (μ\boldsymbol{\mu}, (Σj)j(\boldsymbol{\Sigma}_j)_j, p\boldsymbol{p}) stored in a list column true_theta. Scenario rows are always evaluated sequentially to avoid nested parallelism; sample-level workers live only in deconvolute_ratios(). Scenario builders (factorial grids, overlap summaries, etc.) should live in analysis scripts; see scripts/fig02_bivariate_toy.R and the paper-scenario vignettes.

Usage

run_simulation_benchmark(
  scenario_config,
  deconvolution_functions,
  n = 200,
  standardise = FALSE,
  scaled = FALSE,
  cores = 1L,
  coverage_interval = "wilson",
  verbose = FALSE,
  progress_every = 10L
)

Arguments

scenario_config

Tibble or list of scenario rows. Each row must contain a true_theta list column (or list element) with at least mu and sigma. Optional per-row n overrides the default n.

deconvolution_functions

Named list passed to deconvolute_ratios(); each element has FUN and optional additional_parameters for do.call().

n

Default number of bulk replicates NN when scenario_config has no n column.

standardise, scaled

Passed to deconvolute_ratios().

cores

Workers for the per-sample loop inside deconvolute_ratios(). Defaults to 1L.

coverage_interval

Coverage interval for the Monte Carlo coverage rate; see coverage_mc_interval().

verbose

If TRUE, print each scenario row and (when the grid has at most 10 scenarios) every progress_every inferred samples. Large factorial grids log one line per scenario only, so logs stay readable.

progress_every

Sample-progress interval passed to deconvolute_ratios() when verbose is TRUE. Defaults to 10L.

Value

A list with:

  • regression: global (per-sample composition scores) and cell_type (across-sample Pearson / F1 / spillover);

  • monte_carlo: ADEMP summaries per cell type;

  • optimisation: per-sample elapsed time, memory, KKT residual, and p^\hat{\boldsymbol{p}};

  • config: tibble of scenario metadata (one row per scenario);

  • theta_true: list of convolution parameters (p\boldsymbol{p}, μ\boldsymbol{\mu}, Σj\boldsymbol{\Sigma}_j) actually used to draw the bulk;

  • descriptors: kept scenario statistics (composition, mean, SPD, network, tangent Fisher, MixSim BarOmega, pairwise Hellinger);

  • supplementary: Jeffreys overlap, recorded separately;

  • call: the matched call (match.call()). Analysis scripts persist those pieces with write_simulation_artefacts() (scripts/auxiliary/benchmark_artefacts.R) rather than saving the whole list.

See Also

simulate_bulk_mixture(), deconvolute_ratios(), compute_benchmark_metrics(), describe_simulation_scenario(), coverage_mc_interval(), plot_mc_raincloud(), plot_mc_forest()

Examples

set.seed(1)
genes <- paste0("g", 1:2)
cts <- paste0("ct", 1:2)
mu <- matrix(c(20, 22, 22, 20), nrow = 2, dimnames = list(genes, cts))
Sigma <- array(
  c(1, 0, 0, 1, 1, 0, 0, 1),
  dim = c(2, 2, 2),
  dimnames = list(genes, genes, cts)
)
scenario_config <- tibble::tibble(
  ID = "B1_Ho",
  true_theta = list(list(
    p = c(0.5, 0.5),
    mu = mu,
    sigma = Sigma
  ))
)
out <- run_simulation_benchmark(
  scenario_config = scenario_config,
  deconvolution_functions = list(
    "nnls" = list(FUN = deconvolute_ratios_nnls)
  ),
  n = 2,
  cores = 1
)
nrow(out$optimisation)

Scale component covariances, keeping precision zeros

Description

Replaces each Σj\Sigma_j by sΣjs\Sigma_j. Then Ωj(s)=Ωj/s\Omega_j(s)=\Omega_j/s, so the support of the precision (structural zeros from the graph) is unchanged.

Usage

scale_covariance_array(sigma, scale)

Arguments

sigma

G×G×JG\times G\times J covariance array.

scale

Positive scalar ss.

Value

Scaled array.

See Also

scale_covariances_to_overlap()

Examples

sigma <- array(c(diag(2), diag(2)), dim = c(2, 2, 2))
scaled <- scale_covariance_array(sigma, 4)
scaled[, , 1]

Simulate a bulk sample as a sum of single cells

Description

Draws cell counts m⋅i∼Multinomial(ni,p⋅i)m_{\cdot i}\sim Multinomial(n_i,p_{\cdot i}), then the exact Gaussian sum of those cells, Tji∼N(mjiμ⋅j,mjiΣW,j)T_{ji}\sim N(m_{ji}\mu_{\cdot j},m_{ji}\Sigma_{W,j}), and scales the total by the platform factor: y⋅i=ai∑jTji\boldsymbol{y}_{\cdot i}=a_i\sum_j\boldsymbol{T}_{ji}. A sum of mm independent Gaussian cells is Gaussian, so there is no per-cell loop.

This is the single-cell mixture (covariance linear in the cell counts). simulate_bulk_mixture() remains the population-profile convolution, whose covariance is weighted by pj2p_j^2.

Pass the cellular moments (generative_mean, generative_sigma from rescale_single_cell_moments()), not a library-normalised reference. The returned estimand is the realised cell fraction m⋅i/ni\boldsymbol{m}_{\cdot i}/n_i.

Usage

simulate_bulk_from_cells(
  mean_signature_matrix,
  Sigma,
  p,
  n_cells,
  platform_scale = 1,
  n_samples = NULL,
  seed = NULL
)

Arguments

mean_signature_matrix

Per-cell means μ~\tilde{\boldsymbol{\mu}} (G×JG\times J).

Sigma

Per-cell covariances ΣW,j\boldsymbol{\Sigma}_{W,j} (G×G×JG\times G\times J).

p

Cell-fraction vector of length JJ, or a J×NJ\times N matrix. Each column must sum to 1.

n_cells

Cells per bulk sample, nin_i. Recycled to NN. The multinomial size is the rounded value; the fitted n^\hat n need not be an integer.

platform_scale

Capture scale aia_i. Use 1 when the bulk and the single-cell platform share their scale, and 2 when the bulk captures twice as much transcript per cell. Recycled to NN.

n_samples

Number of bulk samples when p is a vector. Ignored when p already has one column per sample, unless it matches that number.

seed

Optional seed, passed to set.seed() before the draws.

Value

A list with Y (G×NG\times N), nominal p, realised p_realised (m/n\boldsymbol{m}/n), counts (m\boldsymbol{m}), transcript fractions q, cell_size Sj=1⊤μ~⋅jS_j=\mathbf{1}^{\top} \tilde{\boldsymbol{\mu}}_{\cdot j}, platform_scale aa, n_cells nn, mean_scale c=anc=an and variance_scale ϕ=a2n\phi=a^{2}n.

See Also

simulate_bulk_mixture(), rescale_single_cell_moments(), deconvolute_single_cell()

Examples

genes <- paste0("g", 1:4)
cts <- paste0("ct", 1:2)
mu <- matrix(
  c(2, 2, 0.2, 0.2, 0.2, 0.2, 3, 3),
  nrow = 4,
  dimnames = list(genes, cts)
)
sigma <- array(0, c(4, 4, 2), dimnames = list(genes, genes, cts))
sigma[,, 1] <- diag(0.05, 4)
sigma[,, 2] <- diag(0.05, 4)
set.seed(1)
sim <- simulate_bulk_from_cells(
  mu,
  sigma,
  p = c(0.8, 0.2),
  n_cells = 2000,
  platform_scale = 2,
  n_samples = 2
)
colSums(sim$counts)

Simulate bulk mixtures from a multivariate Gaussian convolution

Description

For each bootstrap sample i=1,…,Ni=1,\ldots,N, draws latent purified profiles x⋅j(i)∼NG(μ⋅j,Σj)\boldsymbol{x}_{\cdot j}^{(i)}\sim \mathcal{N}_{G}(\boldsymbol{\mu}_{\cdot j}, \boldsymbol{\Sigma}_j) independently for each cell type j=1,…,Jj=1,\ldots,J, then forms the bulk by the linear convolution

y⋅i=X(i)p=∑j=1Jpj x⋅j(i),\boldsymbol{y}_{\cdot i} =\boldsymbol{X}^{(i)}\boldsymbol{p} =\sum_{j=1}^{J}p_j\,\boldsymbol{x}_{\cdot j}^{(i)},

matching the article's conditional model y ∣ (ζ,p)∼NG(μp,Σ(p))\boldsymbol{y}\,|\,(\boldsymbol{\zeta},\boldsymbol{p})\sim \mathcal{N}_{G}(\boldsymbol{\mu}\boldsymbol{p}, \boldsymbol{\Sigma}(\boldsymbol{p})) with Σ(p)=∑jpj2Σj\boldsymbol{\Sigma}(\boldsymbol{p})= \sum_j p_j^{2}\boldsymbol{\Sigma}_j.

The mean signature μ\boldsymbol{\mu} is shared across samples; only the latent draws x⋅j(i)\boldsymbol{x}_{\cdot j}^{(i)} vary with ii. Stacking those draws into the three-way array X=(xgji)∈MG×J×N\mathcal{X}=(x_{gji})\in\mathcal{M}_{G\times J\times N}, the bulk matrix is the mode-2 tensor–vector contraction

Y=X×2p,ygi=∑j=1Jxgji pj(g=1,…,G;  i=1,…,N),\boldsymbol{Y} =\mathcal{X}\times_{2}\boldsymbol{p}, \qquad y_{gi}=\sum_{j=1}^{J}x_{gji}\,p_{j} \quad(g=1,\ldots,G;\; i=1,\ldots,N),

which for each sample recovers y⋅i=X(i) p\boldsymbol{y}_{\cdot i}=\boldsymbol{X}^{(i)}\,\boldsymbol{p} with X(i)∈MG×J\boldsymbol{X}^{(i)}\in\mathcal{M}_{G\times J}.

Usage

simulate_bulk_mixture(
  signature_matrix,
  Sigma,
  p = rep(1/ncol(signature_matrix), ncol(signature_matrix)),
  n = 500,
  truncate_negative = FALSE
)

Arguments

signature_matrix

Mean signature μ=(μgj)∈MG×J\boldsymbol{\mu}=(\mu_{gj})\in\mathcal{M}_{G\times J} (shared across samples; not the latent profiles).

Sigma

Array of covariances (Σj)j∈MG×G×J(\boldsymbol{\Sigma}_j)_{j}\in\mathcal{M}_{G\times G\times J}.

p

Proportion vector p∈ΔJ−1\boldsymbol{p}\in\Delta^{J-1} (default: uniform).

n

Number of bulk / bootstrap samples NN.

truncate_negative

If TRUE, replace negative bulk entries with 0 so Y\boldsymbol{Y} is a valid RNA-seq-style non-negative matrix. The latent Gaussian draws are unchanged. deconvolute_ratios() allows negative bulk entries (with a warning): Gaussian convolutions can produce them. Flooring is optional, not required.

Value

A list with:

  • latent_profiles: array X=(xgji)∈MG×J×N\mathcal{X}=(x_{gji})\in\mathcal{M}_{G\times J\times N} of unobserved cell-type-specific draws (one G×JG\times J slice per sample ii);

  • Y: matrix Y∈MG×N\boldsymbol{Y}\in\mathcal{M}_{G\times N} whose columns are bulk vectors y⋅i\boldsymbol{y}_{\cdot i}, obtained as X×2p\mathcal{X}\times_{2}\boldsymbol{p}.

See Also

deconvolute_ratios(), run_simulation_benchmark(), simulate_bulk_from_cells()

Examples

set.seed(1)
genes <- paste0("g", 1:2)
cts <- paste0("ct", 1:2)
mu <- matrix(c(20, 22, 22, 20), nrow = 2, dimnames = list(genes, cts))
Sigma <- array(
  c(1, 0, 0, 1, 1, 0, 0, 1),
  dim = c(2, 2, 2),
  dimnames = list(genes, genes, cts)
)
sim <- simulate_bulk_mixture(mu, Sigma, p = c(0.5, 0.5), n = 5)
dim(sim$Y)

Simulate GRN first- and second-order moments

Description

Builds a mean matrix μ∈MG×J\boldsymbol{\mu}\in\mathcal{M}_{G\times J} and cell-type-specific second-order moments (Ωj,Σj=Ωj−1)j=1J(\boldsymbol{\Omega}_j, \boldsymbol{\Sigma}_j=\boldsymbol{\Omega}_j^{-1})_{j=1}^{J} under a graph-constrained precision model. Means realise a target Gram matrix via the symmetric square root of generate_mean_signature_matrix() (equicorrelation ρ\rho by default, or a full target_gram). For each cell type, an adjacency is drawn from a random-graph model (or supplied), i.i.d. signed weights with inhibitory fraction prop_inhibitory form Wj\boldsymbol{W}_j, and the precision is completed by a spectral shift. Distinct cell types receive independent precision draws by default (biology rarely shares one network across types); pass a length-JJ graph_model / graph_params or a pre-built adjacency list / array for hybrid designs.

Usage

simulate_hierarchical_grn_moments(
  n_genes,
  n_celltypes = 2L,
  mean_scale = 10,
  target_cosine = 0,
  target_gram = NULL,
  seed = NULL,
  nonnegative = FALSE,
  precision_shift,
  precision_scale,
  prop_inhibitory = 0.5,
  graph_model = c("erdos_renyi", "hub", "star", "scale_free", "stochastic_block_model",
    "small_world"),
  graph_params = list(),
  adjacency = NULL
)

Arguments

n_genes

Integer GG; must be at least n_celltypes.

n_celltypes

Integer J≥2J\ge 2.

mean_scale

Positive scalar ss (centroid norms). Default 10. Hold fixed when studying cosine / collinearity alone.

target_cosine

Numeric pairwise cosine ρ\rho for the equicorrelation Gram. Ignored when target_gram is supplied. Must lie in [−1/(J−1),1][-1/(J-1),1].

target_gram

Optional symmetric J×JJ\times J cosine matrix (unit diagonal). Overrides target_cosine.

seed

Optional integer. When supplied, the orthonormal frame Q\boldsymbol{Q} is drawn from a Gaussian QR; otherwise it is deterministic. Ignored when nonnegative = TRUE.

nonnegative

If TRUE, use a disjoint-support nonnegative frame so that μ≥0\boldsymbol{\mu}\ge 0 whenever R1/2≥0R^{1/2}\ge 0 (required by deconvolute_ratios()). The Gram is unchanged.

precision_shift

Diagonal cushion uu for the spectral shift (scalar, or length JJ).

precision_scale

Positive magnitude vv of signed off-diagonal precision weights (scalar, or length JJ).

prop_inhibitory

Numeric in [0,1][0,1]; fraction of edges with positive precision weight (inhibitory partial correlation). Default 0.5 balances inhibitory and activatory edges (scalar, or length JJ).

graph_model

One of "erdos_renyi", "hub", "scale_free", "stochastic_block_model", "small_world", or a character vector of length JJ.

graph_params

Named list of generator parameters (shared), or a list of length JJ of such named lists (see generate_random_network_skeleton()).

adjacency

Optional pre-built undirected adjacencies: a list of JJ G×GG\times G matrices, or a G×G×JG\times G\times J array. When supplied, graph_model / graph_params are ignored for skeleton generation.

Value

Named list with:

  • mean_profiles: matrix μ\boldsymbol{\mu};

  • covariance_matrices: array (Σj)j∈MG×G×J(\boldsymbol{\Sigma}_j)_{j}\in\mathcal{M}_{G\times G\times J};

  • precision_matrices: array (Ωj)j∈MG×G×J(\boldsymbol{\Omega}_j)_{j}\in\mathcal{M}_{G\times G\times J};

  • graph_structure: adjacency_matrices, weighted_adjacencies, and normalised_precision (all G×G×JG\times G\times J);

  • objectives: mean_abs_cosine and sum_euclidean_distance.

Examples

set.seed(42)
moments <- simulate_hierarchical_grn_moments(
  n_genes = 40L,
  n_celltypes = 3L,
  mean_scale = 10,
  target_cosine = 0.1,
  precision_shift = 0.1,
  precision_scale = 0.3,
  prop_inhibitory = 0.5,
  graph_model = "scale_free"
)
str(moments, max.level = 2)

Stage-i scales for one single-cell bulk sample

Description

Generalised method of moments for the two technical scales, at a composition that is either supplied or taken from the free-scale DeconRNASeq start (c(0),p(0))=arg⁡min⁡c>0, p∈Δ∥y−cμ~p∥22(c^{(0)},\boldsymbol{p}^{(0)}) =\arg\min_{c>0,\,p\in\Delta}\lVert y-c\tilde\mu p\rVert_2^2. The start is floored into the open simplex. At that composition, c^=(m⊤Wm)−1m⊤Wy\hat c =(\boldsymbol{m}^{\top}\boldsymbol{W}\boldsymbol{m})^{-1} \boldsymbol{m}^{\top}\boldsymbol{W}\boldsymbol{y} and ϕ^=r⊤Wr/G\hat\phi=\boldsymbol{r}^{\top}\boldsymbol{W}\boldsymbol{r}/G, with W=SW(p)−1\boldsymbol{W}=\boldsymbol{S}_W(\boldsymbol{p})^{-1}. Then a^=ϕ^/c^\hat a=\hat\phi/\hat c and n^=c^2/ϕ^\hat n=\hat c^2/\hat\phi. n^\hat n is a cell count only when the two platforms share their scale; it is not rounded.

c^\hat c is not the library-size ratio 1⊤y/∑jpjSj\mathbf{1}^{\top}y/\sum_j p_j S_j.

Usage

single_cell_nuisance_scales(y, mean_signature_matrix, Sigma, p = NULL)

Arguments

y

Bulk vector of length GG.

mean_signature_matrix

Reference mean, G×JG\times J.

Sigma

Cell-level covariances, G×G×JG\times G\times J.

p

Optional composition. NULL runs the free-scale start.

Value

List with p, ols_scale (the least-squares c(0)c^{(0)}, or NA when p was supplied), mean_scale, variance_scale, platform_scale and n_cells.

See Also

deconvolute_single_cell(), deconvolute_ratios_deconrnaseq()

Examples

mu <- matrix(c(2, 0.2, 0.2, 2, 0.2, 3, 0.2, 3), nrow = 4)
sigma <- array(rep(diag(0.05, 4), 2), c(4, 4, 2))
set.seed(1)
y <- 30 * drop(mu %*% c(0.6, 0.4)) + rnorm(4, sd = 0.5)
single_cell_nuisance_scales(y, mu, sigma)

Four single-cell simulation scenarios

Description

The scenario grid used to check the single-cell fit. Two compositions, equal and imbalanced, crossed with equal and unequal cell sizes. Every scenario is drawn at G=10G=10 genes, ni=104n_i=10^{4} cells and platform scales ai∈{1,2}a_i\in\{1, 2\} (same capture efficiency, or twice the bulk capture of the single-cell platform).

Usage

single_cell_scenario_design()

Value

A list with n_genes, n_cells, platform_scale and scenarios (four named lists with proportions and cell_size).

See Also

rescale_single_cell_moments(), simulate_bulk_from_cells()

Examples

names(single_cell_scenario_design()$scenarios)

Open-simplex start for ILR solvers

Description

The convolution log-likelihood is not globally concave, so the start can change which basin the solver enters. Draw several independent Dirichlet starts (different RNG streams or multistart_decovart()) rather than a single draw when probing multimodality.

Usage

starting_simplex(
  n_celltypes,
  initial_p = NULL,
  nms = NULL,
  y = NULL,
  mean_signature_matrix = NULL,
  dirichlet_alpha = 1
)

Arguments

n_celltypes

Integer JJ.

initial_p

One of:

  • NULL or "barycentre": equi-balanced (1/J,…,1/J)(1/J,\ldots,1/J);

  • a numeric vector of length JJ: used as-is after repair_simplex();

  • "dirichlet": one Dirichlet(α,…,α)(\alpha,\ldots,\alpha) draw (see dirichlet_alpha);

  • "qp" (aliases "deconrnaseq", "lsei"): mean-only simplex QP from deconvolute_ratios_deconrnaseq() (y and mean_signature_matrix required).

nms

Optional names (cell-type colnames).

y, mean_signature_matrix

Bulk and signature, required for a QP start.

dirichlet_alpha

Positive concentration, recycled to length JJ. The default 1 is uniform on the simplex. α>1\alpha>1 concentrates mass near the barycentre; α<1\alpha<1 puts extra mass near faces (boundary-biased restarts).

Value

A length-JJ open-simplex vector.

See Also

multistart_decovart(), deconvolute_ratios_deconrnaseq()

Examples

starting_simplex(3L)
set.seed(1)
starting_simplex(3L, "dirichlet")

Faceted ggplot2 theme (black strips, panel border)

Description

theme_minimal() plus a panel border and white-on-black facet strips, after theme_features() in the atlas-feature-selection-benchmark plotting helpers.

Usage

theme_decovart_facets(base_size = 11, ...)

Arguments

base_size

Base font size for ggplot2::theme_minimal().

...

Passed to ggplot2::theme().

Value

A ggplot2 theme object.

See Also

plot_mc_forest(), plot_mc_raincloud()

Examples

ggplot2::ggplot(mtcars, ggplot2::aes(wt, mpg)) +
  ggplot2::geom_point() +
  ggplot2::facet_wrap(~cyl) +
  theme_decovart_facets()

Cramer–Rao / ILR delta-method covariance of p^\hat{\boldsymbol{p}}

Description

Maps the expected Fisher information of unconstrained proportions through the isometric log-ratio (ILR) chart and back to the simplex via the delta method.

Let p=softmax⁡(Vz)\boldsymbol{p}=\operatorname{softmax}(\mathbf{V}\boldsymbol{z}) with Jacobian Jψ=S(p)V\mathbf{J}_{\boldsymbol{\psi}}=\mathbf{S}(\boldsymbol{p}) \mathbf{V} (jacobian_isometric_logistic()). Fisher information transforms as

Iz=JψTI(p)Jψ=VTS(p)I(p)S(p)V.I_{\boldsymbol{z}} = \mathbf{J}_{\boldsymbol{\psi}}^{\mathsf{T}} I(\boldsymbol{p}) \mathbf{J}_{\boldsymbol{\psi}} = \mathbf{V}^{\mathsf{T}} \mathbf{S}(\boldsymbol{p}) I(\boldsymbol{p}) \mathbf{S}(\boldsymbol{p}) \mathbf{V}.

The first-order delta method then yields

Var(p^)≈JψIz−1JψT.\mathrm{Var}(\hat{\boldsymbol{p}}) \approx \mathbf{J}_{\boldsymbol{\psi}} I_{\boldsymbol{z}}^{-1} \mathbf{J}_{\boldsymbol{\psi}}^{\mathsf{T}}.

This simplex covariance is invariant to orthogonal rotations of V\mathbf{V}. The construction is undefined on the simplex boundary (the log-ratio chart blows up); the function then returns NA with a warning.

Wald intervals in vcov.decovart_fit(), confint.decovart_fit() and deconvolute_ratios() all use this expected-Fisher ILR map. The observed Hessian hessian_loglik_constrained() is for Newton steps only; it is not the Wald covariance.

Usage

vcov_ilr_delta(p, mean_signature_matrix, Sigma, warn = TRUE)

Arguments

p

Numeric proportions on the open simplex.

mean_signature_matrix

Mean signature μ\boldsymbol{\mu} (G×JG\times J).

Sigma

Cell-type covariances G×G×JG\times G\times J.

warn

If FALSE, skip the boundary / singularity warnings (Monte Carlo loops). Default TRUE.

Value

Symmetric J×JJ\times J asymptotic covariance of p^\hat{\boldsymbol{p}}, or a matrix of NA if the bound is undefined / singular.

See Also

expected_fisher_unconstrained(), vcov.decovart_fit(), confint.decovart_fit(), jacobian_isometric_logistic()

Examples

# Two cell types: a single unconstrained ILR coordinate maps back to
# a rank-1 covariance on the simplex (rows sum to zero).
p <- c(ct1 = 0.6, ct2 = 0.4)
mu <- cbind(ct1 = c(0, 0), ct2 = c(3, 0))
Sigma <- array(c(diag(2), diag(2)), dim = c(2, 2, 2))
v <- vcov_ilr_delta(p, mu, Sigma)
round(v, 4)
abs(rowSums(v)) < 1e-10

Observed information of one single-cell fit

Description

At a fitted composition, the covariance of the ILR coordinate ρ^\hat{\boldsymbol{\rho}} is the inverse of the negative profile Hessian hessian_profile_loglik_single_cell_ilr(). The delta method carries it to p^\hat{\boldsymbol{p}} with Jψ\mathbf{J}_{\psi}. The two technical scales use the other Schur complement of the joint observed Hessian in (ρ,c,ϕ)(\boldsymbol{\rho},c,\phi): estimating the composition widens their variances, and IcϕI_{c\phi} is not assumed to make that complement diagonal.

Both matrices are observed information, not expected Fisher information. They treat the reference moments as known.

Usage

vcov_single_cell(
  p,
  y,
  mean_signature_matrix,
  Sigma,
  mean_scale = NULL,
  variance_scale = NULL,
  scales_known = FALSE,
  warn = TRUE
)

Arguments

p

Fitted composition. A boundary coordinate makes the ILR chart undefined.

y

Bulk vector of length GG.

mean_signature_matrix

Reference mean, G×JG\times J.

Sigma

Cell-level covariances, G×G×JG\times G\times J.

mean_scale, variance_scale

Scales at p. NULL recomputes the closed forms.

scales_known

If TRUE, the scales are treated as known: the composition covariance inverts the fixed-scale observed Hessian hessian_loglik_single_cell_ilr() at mean_scale and variance_scale, and vcov_nuisance is NA. This is the covariance of a "fixed_scales" fit. For a "two_stage" fit it ignores the uncertainty of the start scales, so its intervals are too narrow.

warn

If TRUE, warn when either information matrix is not positive definite.

Value

A list with rho, vcov_rho, vcov (delta method for p\boldsymbol{p}), mean_scale, variance_scale and vcov_nuisance (order cc, ϕ\phi). Singular blocks are filled with NA.

See Also

deconvolute_single_cell(), confint_single_cell()

Examples

mu <- matrix(c(2, 0.2, 0.2, 2, 0.2, 3, 0.2, 3), nrow = 4)
sigma <- array(rep(diag(0.05, 4), 2), c(4, 4, 2))
y <- 20 * drop(mu %*% c(0.7, 0.3)) + c(0.3, -0.2, 0.1, -0.1)
vcov_single_cell(c(0.7, 0.3), y, mu, sigma)