Dimensionality Estimation Module

Dimensionality estimation module for DRIADA.

This module provides various methods for estimating the intrinsic dimensionality of datasets, including both linear and nonlinear approaches.

Methods for estimating the intrinsic dimensionality of high-dimensional datasets, including linear methods (PCA-based), effective dimensionality, and nonlinear approaches.

Module Components

• Linear Dimensionality Methods
• Effective Dimensionality Estimation
• Intrinsic Dimensionality Estimation

Quick Links

Linear Methods

• pca_dimension() - PCA variance threshold

• pca_dimension_profile() - Full PCA profile

• effective_rank() - Matrix effective rank

• Linear Dimensionality Methods - All linear methods

Effective Dimensionality

• eff_dim() - Participation ratio method

• Effective Dimensionality Estimation - Rényi entropy-based estimation

Intrinsic Dimensionality

• nn_dimension() - k-NN based estimation

• correlation_dimension() - Correlation dimension

• geodesic_dimension() - Geodesic-based estimation

• Intrinsic Dimensionality Estimation - Nonlinear methods

Usage Example

from driada.dimensionality import (
    pca_dimension, eff_dim, nn_dimension
)
from driada.experiment import load_demo_experiment

# Load sample experiment
exp = load_demo_experiment()

# Prepare data (n_samples, n_features)
neural_data = exp.calcium.scdata.T

# Add small noise to avoid duplicate points for nn_dimension
import numpy as np
neural_data_noisy = neural_data + 1e-6 * np.random.randn(*neural_data.shape)

# Linear methods
pca_dim_90 = pca_dimension(neural_data, threshold=0.90)
pca_dim_95 = pca_dimension(neural_data, threshold=0.95)

# Effective dimension (participation ratio)
eff_d = eff_dim(neural_data, enable_correction=True)

# Nonlinear intrinsic dimension (use noisy data to avoid duplicates)
nn_dim = nn_dimension(neural_data_noisy, k=5)

print(f"PCA 90%: {pca_dim_90}, PCA 95%: {pca_dim_95}")
print(f"Effective dim: {eff_d:.2f}")
print(f"k-NN dimension: {nn_dim:.2f}")