Quick start guide

This guide will help you get started with DRIADA in just a few minutes.

Installation

# Basic installation
pip install driada

# With GPU support (recommended for large datasets)
pip install driada[gpu]

For an interactive introduction, start with the DRIADA overview notebook which covers data loading, feature types, and demos of each analysis module.

Getting started with DRIADA

1. Generate synthetic data for testing

Start by generating synthetic data to understand DRIADA’s capabilities:

import driada
from driada.experiment import generate_circular_manifold_exp
import numpy as np

# Generate a population with head direction cells
exp = generate_circular_manifold_exp(
    n_neurons=50,           # 50 head direction cells
    duration=60,            # 1 minute of recording (for demo)
    noise_std=0.1,          # 10% noise (std deviation)
    seed=42
)

# Or generate place cells in 2D environment
from driada.experiment import generate_2d_manifold_exp

exp = generate_2d_manifold_exp(
    n_neurons=64,           # 8x8 grid of place cells
    duration=900,           # 15 minutes of exploration
    fps=20.0,               # Frame rate
    seed=42
)

# Or create mixed populations
from driada.experiment import generate_mixed_population_exp

exp = generate_mixed_population_exp(
    n_neurons=100,
    manifold_type='circular',
    manifold_fraction=0.4,  # 40% manifold cells, 60% feature-selective
    duration=600,
    seed=42
)

2. Analyze single-neuron selectivity (INTENSE)

Discover which neurons encode which variables:

# Discover which neurons encode which variables
from driada.intense import compute_cell_feat_significance
from driada.experiment import load_demo_experiment

exp = load_demo_experiment()  # Load sample experiment
stats, significance, info, results = compute_cell_feat_significance(
    exp,
    n_shuffles_stage1=100,    # Quick screening
    n_shuffles_stage2=1000,   # Rigorous validation
    ds=5,                     # Downsample by factor of 5 for speed
    verbose=True,
    find_optimal_delays=False # Skip temporal alignment for demo
)

# View results
significant_neurons = exp.get_significant_neurons()
print(f"Found {len(significant_neurons)} selective neurons")

# Visualize selectivity
if significant_neurons:
    from driada.intense.visual import plot_neuron_feature_pair

    neuron_id = list(significant_neurons.keys())[0]
    feature_name = significant_neurons[neuron_id][0]
    plot_neuron_feature_pair(exp, neuron_id, feature_name)

3. Estimate intrinsic dimensionality

Before applying dimensionality reduction, estimate the intrinsic dimensionality:

# Multiple methods for dimensionality estimation
from driada.dimensionality import (
    eff_dim, pca_dimension, nn_dimension, correlation_dimension
)
from driada.experiment import load_demo_experiment

# Load sample experiment
exp = load_demo_experiment()

# Get neural activity data (n_samples, n_features)
neural_data = exp.calcium.scdata.T  # Transpose to standard format

# Linear methods
pca_90 = pca_dimension(neural_data, threshold=0.90)
pca_95 = pca_dimension(neural_data, threshold=0.95)

# Effective dimension (participation ratio)
eff_d = eff_dim(neural_data, enable_correction=True, q=2)

# Nonlinear methods - add small noise to avoid duplicate points
neural_data_noisy = neural_data + 1e-8 * np.random.randn(*neural_data.shape)
nn_dim = nn_dimension(neural_data_noisy, k=5)
corr_dim = correlation_dimension(neural_data_noisy)

print(f"PCA 90%: {pca_90} dims, PCA 95%: {pca_95} dims")
print(f"Effective dim: {eff_d:.2f}")
print(f"k-NN dimension: {nn_dim:.2f}")
print(f"Correlation dimension: {corr_dim:.2f}")

4. Apply dimensionality reduction

Extract low-dimensional representations of population activity:

from driada.experiment import load_demo_experiment

# Load sample experiment
exp = load_demo_experiment()

# Two approaches for dimensionality reduction:
# 1. Direct method: exp.calcium.get_embedding() returns Embedding objects
# 2. Experiment method: exp.create_embedding() returns numpy arrays and stores them

# Approach 1: Direct dimensionality reduction on calcium data
# exp.calcium is a MultiTimeSeries, which inherits from MVData
# So it directly supports all dimensionality reduction methods!

# Apply different dimensionality reduction methods directly on calcium data
# PCA - captures linear variance
pca_emb = exp.calcium.get_embedding(method='pca', dim=3)

# Isomap - preserves geodesic distances
iso_emb = exp.calcium.get_embedding(method='isomap', dim=2, n_neighbors=30)

# UMAP - preserves local and global structure
umap_emb = exp.calcium.get_embedding(method='umap', dim=2,
                                    n_neighbors=50, min_dist=0.1)

# t-SNE - emphasizes local structure
tsne_emb = exp.calcium.get_embedding(method='tsne', dim=2, perplexity=30)

# Access the coordinates from the Embedding object
coords = pca_emb.coords.T  # (n_samples, n_dims)

# For custom downsampling, create new MVData
from driada.dim_reduction import MVData
mvdata_ds = MVData(exp.calcium.data, downsampling=5)
pca_ds = mvdata_ds.get_embedding(method='pca', dim=3)

# Approach 2: Use experiment's create_embedding() to store embeddings
# This is required for INTENSE analysis (compute_embedding_selectivity)
# Returns numpy arrays instead of Embedding objects
pca_array = exp.create_embedding('pca', n_components=3)
umap_array = exp.create_embedding('umap', n_components=2)

# These are now stored in exp.embeddings and can be retrieved:
stored_pca = exp.get_embedding('pca')  # Returns dict with 'data' key
print(f"Stored PCA shape: {stored_pca['data'].shape}")

5. Validate manifold quality

Assess how well the embedding preserves the original structure:

from driada.dim_reduction import (
    knn_preservation_rate, trustworthiness, continuity
)
from driada.experiment import load_demo_experiment

# Load sample experiment and create embedding
exp = load_demo_experiment()
pca_emb = exp.calcium.get_embedding(method='pca', dim=3)

# Compare high-D and low-D representations
# Need to transpose calcium data to get (n_samples, n_features)
high_d = exp.calcium.scdata.T  # Original high-dimensional data
low_d = pca_emb.coords.T        # Low-dimensional embedding from above

# k-NN preservation: how many neighbors stay the same
knn_score = knn_preservation_rate(high_d, low_d, k=10)

# Trustworthiness: are close points in low-D truly close in high-D?
trust = trustworthiness(high_d, low_d, k=10)

# Continuity: are close points in high-D still close in low-D?
cont = continuity(high_d, low_d, k=10)

print(f"k-NN preservation: {knn_score:.3f}")
print(f"Trustworthiness: {trust:.3f}")
print(f"Continuity: {cont:.3f}")

6. Integrate single-cell and population analysis

Analyze how single neurons contribute to population embeddings:

# First, compute INTENSE selectivity for embedding components
from driada.intense import compute_embedding_selectivity
from driada.experiment import load_demo_experiment

# Load sample experiment
exp = load_demo_experiment()

# First, store embeddings using create_embedding() if not already done
# This returns numpy arrays and stores them for INTENSE analysis
if 'pca' not in exp.embeddings['calcium']:
    exp.create_embedding('pca', n_components=3)
if 'umap' not in exp.embeddings['calcium']:
    exp.create_embedding('umap', n_components=2)

# Then analyze how neurons contribute to embedding components
emb_results = compute_embedding_selectivity(
    exp,
    embedding_methods=['pca', 'umap'],
    n_shuffles_stage1=100,
    n_shuffles_stage2=1000,
    find_optimal_delays=False  # Skip temporal alignment for demo
)

# Extract INTENSE results for functional organization analysis
from driada.integration import get_functional_organization

# Analyze PCA functional organization
pca_org = get_functional_organization(
    exp,
    'pca',
    intense_results=emb_results['pca']['intense_results']
)

print(f"Component importance: {pca_org['component_importance']}")
print(f"Neurons participating: {pca_org['n_participating_neurons']}")

# Compare multiple embeddings
from driada.integration import compare_embeddings

intense_dict = {
    'pca': emb_results['pca']['intense_results'],
    'umap': emb_results['umap']['intense_results']
}

comparison = compare_embeddings(
    exp,
    ['pca', 'umap'],
    intense_results_dict=intense_dict
)

# Visualize embeddings with features
from driada.utils.visual import plot_embedding_comparison

# Load experiment and create embeddings if needed
exp = load_demo_experiment()

# First create the embeddings if not already done
if 'pca' not in exp.embeddings['calcium']:
    exp.create_embedding('pca', n_components=3)
if 'umap' not in exp.embeddings['calcium']:
    exp.create_embedding('umap', n_components=2)

# Get the stored numpy arrays from the experiment
embeddings = {
    'PCA': exp.get_embedding('pca')['data'],  # Get stored numpy array
    'UMAP': exp.get_embedding('umap')['data']  # Get stored numpy array
}

# Color by a behavioral feature (ensure lengths match)
features = {}
if 'position_2d' in exp.dynamic_features:
    pos = exp.dynamic_features['position_2d'].data
    angle = np.arctan2(pos[1] - 0.5, pos[0] - 0.5)
    # Handle downsampling if embeddings were downsampled
    if hasattr(exp.calcium, 'downsampling'):
        ds = exp.calcium.downsampling
        features['angle'] = angle[::ds]
    else:
        features['angle'] = angle

fig = plot_embedding_comparison(
    embeddings=embeddings,
    features=features,
    compute_metrics=True,
    figsize=(12, 5)
)

7. Network analysis

The Network class provides general-purpose graph analysis – spectral decomposition, entropy, community detection, and visualization for any graph. Within DRIADA, it integrates naturally with INTENSE (cell-cell significance produces adjacency matrices) and dimensionality reduction (graph-based methods construct a ProximityGraph that inherits from Network).

Here is an example using INTENSE cell-cell connectivity:

from driada.intense import compute_cell_cell_significance
from driada.network import Network
from driada.experiment import load_demo_experiment
import scipy.sparse as sp

# Load sample experiment
exp = load_demo_experiment()

# Compute pairwise functional connectivity
# Uses mutual information to measure dependencies
results = compute_cell_cell_significance(
    exp,
    n_shuffles_stage1=100,    # Quick screening
    n_shuffles_stage2=1000,   # Rigorous validation
    ds=5,                     # Downsample for speed
    verbose=True
)

sim_mat, sig_mat, pval_mat, cells, info = results

# sig_mat is binary: 1 = significant correlation, 0 = not significant
n_connections = np.sum(sig_mat)
print(f"Found {n_connections} significant connections")
print(f"Network density: {n_connections / (len(cells)**2 - len(cells)):.3f}")

# Create network from significant connections
sig_sparse = sp.csr_matrix(sig_mat)
net = Network(adj=sig_sparse, preprocessing='giant_cc')

# Analyze network properties
print(f"Network has {net.n} nodes in giant component")
print(f"Average degree: {net.deg.mean():.2f}")

# Detect functional modules
from sklearn.cluster import SpectralClustering

if net.n > 10:
    # Use spectral clustering on the network
    clustering = SpectralClustering(
        n_clusters=3,
        affinity='precomputed',
        random_state=42
    )
    modules = clustering.fit_predict(net.adj.toarray())

    print(f"Detected {len(np.unique(modules))} functional modules")

# Visualize network (for smaller networks)
if net.n < 50:
    import networkx as nx
    import matplotlib.pyplot as plt

    G = nx.from_scipy_sparse_array(net.adj)
    pos = nx.spring_layout(G, seed=42)

    plt.figure(figsize=(10, 8))
    nx.draw_networkx_nodes(G, pos, node_size=300,
                           node_color='lightblue', alpha=0.7)
    nx.draw_networkx_edges(G, pos, alpha=0.5)
    nx.draw_networkx_labels(G, pos, font_size=8)
    plt.title(f"Functional Network ({net.n} neurons)")
    plt.axis('off')
    plt.tight_layout()

8. Working with real data

Load and analyze your own neural recordings:

import numpy as np
from driada import load_exp_from_aligned_data

# Load data from NPZ file (recommended format)
sample_npz_path = 'path/to/sample_recording.npz'
data = dict(np.load(sample_npz_path))
# Expected structure in sample_recording.npz:
# - data['calcium']: (50, 10000) - neural activity, REQUIRED
# - data['position']: (2, 10000) - 2D trajectory (x,y coordinates)
# - data['x_pos']: (10000,) - x coordinate
# - data['y_pos']: (10000,) - y coordinate
# - data['speed']: (10000,) - movement speed
# - data['trial_type']: (10000,) - discrete labels ('A', 'B', 'C', 'D')
# - data['head_direction']: (10000,) - circular variable (radians)
# - data['fps']: scalar - frame rate (30.0)

# Create experiment with automatic feature detection
exp = load_exp_from_aligned_data(
    data_source='MyLab',  # Can be any lab identifier (e.g., 'MyLab', 'NeuroLab')
    exp_params={'name': 'my_experiment'},  # For custom labs, use 'name' key
    data=data,
    static_features={'fps': 30.0},  # Recording frame rate
    force_continuous=['trial_type'],  # Override auto-detection if needed
    bad_frames=[100, 101, 102],  # Mark corrupted frames
    reconstruct_spikes='wavelet'  # Automatic spike deconvolution
)

# For HDF5 files (example with your own file)
# from driada.utils.data import read_hdf5_to_dict
# data = read_hdf5_to_dict('path/to/your/recording.h5')
# exp = load_exp_from_aligned_data(
#     data_source='my_lab',
#     exp_params={'name': 'my_experiment'},
#     data=data
# )

# For multi-dimensional features (e.g., 2D position)
from driada.information.info_base import MultiTimeSeries

# Combine x,y coordinates into single feature
spatial_data = np.stack([data['x_pos'], data['y_pos']])
spatial_feature = MultiTimeSeries(
    spatial_data,
    discrete=False
)

# Add to data dictionary
data['position_2d'] = spatial_feature
exp = load_exp_from_aligned_data(
    data_source='my_lab',
    exp_params={'name': 'my_experiment'},
    data=data
)

9. Advanced analysis workflows

Leverage DRIADA’s advanced capabilities:

# Sequential dimensionality reduction pipeline
from driada.dim_reduction import dr_sequence
from driada.experiment import load_demo_experiment

exp = load_demo_experiment()

# Chain multiple dimensionality reduction methods for optimal results
embedding = dr_sequence(
    exp.calcium,
    steps=[
        ('pca', {'dim': 20}),     # Initial denoising
        ('umap', {'dim': 3, 'n_neighbors': 15})  # Final embedding
    ],
    keep_intermediate=True  # Access results from each step
)

# Access intermediate results (returns tuple when keep_intermediate=True)
final_embedding, intermediate_results = embedding
pca_result = intermediate_results[0]
umap_result = intermediate_results[1]
final_coords = final_embedding.coords.T

# High-precision INTENSE analysis with mixed features
from driada.intense import compute_cell_feat_significance

results = compute_cell_feat_significance(
    exp,
    mode='two_stage',
    n_shuffles_stage1=100,     # Pre-screening
    n_shuffles_stage2=5000,    # High precision
    find_optimal_delays=False,  # Skip temporal alignment
    ds=5,  # Downsample for speed
    verbose=True
)

# Save complete analysis results
import pickle

# Package all results
analysis_results = {
    'experiment_signature': exp.signature,
    'intense_stats': results[0],
    'intense_significance': results[1],
    'embedding_coords': {
        'pca': pca_result.coords.T,  # Convert to numpy array
        'umap': final_embedding.coords.T  # Convert to numpy array
    },
    'significant_neurons': exp.get_significant_neurons()
}

# Save to pickle
with open('analysis_results.pkl', 'wb') as f:
    pickle.dump(analysis_results, f)

Next steps

Explore comprehensive examples demonstrating real-world workflows:

Getting started:

  • examples/data_loading/load_data_example.py - Load your own data into DRIADA

  • examples/neuron_basic_usage/neuron_basic_usage.py - Single neuron creation and spike reconstruction

  • examples/intense_basic_usage/intense_basic_usage.py - Detect selective neurons with INTENSE

Core analysis workflows:

  • examples/circular_manifold/extract_circular_manifold.py - Extract ring attractor from head direction cells

  • examples/spatial_analysis/spatial_visualization.py - Visualize spatial data (trajectory, rate maps, calcium traces)

  • examples/network_analysis/cell_cell_network_example.py - Build and analyze functional networks

  • examples/rsa/rsa_example.py - Representational similarity analysis

Dimensionality reduction:

  • examples/compare_dr_methods/compare_dr_methods.py - Systematic comparison of dimensionality reduction algorithms

  • examples/dr_sequence/dr_sequence_neural_example.py - Sequential dimensionality reduction pipeline

  • examples/dr_simplified_api/dr_simplified_api_demo.py - Simple dimensionality reduction API with all supported methods

Recurrence analysis:

  • examples/recurrence_basic/recurrence_basic.py - Recurrence plots, RQA, visibility and ordinal graphs

  • examples/recurrence_population/recurrence_population.py - Recover functional modules from population dynamics

Complete pipelines:

  • examples/full_intense_pipeline/full_intense_pipeline.py - Full INTENSE workflow with disentanglement

  • examples/intense_dr_pipeline/intense_dr_pipeline.py - INTENSE + dimensionality reduction integration

  • examples/mixed_selectivity/mixed_selectivity.py - Analyze neurons with mixed feature selectivity

  • examples/loo_dr_analysis/loo_dr_analysis.py - Leave-one-out neuron importance for manifolds

Advanced techniques:

  • examples/spike_reconstruction/spike_reconstruction_comparison.py - Compare spike deconvolution methods

  • examples/spike_reconstruction/threshold_vs_wavelet_optimization.py - Iterative detection and kinetics optimization

  • examples/visual_utils/visual_utils_demo.py - Publication-ready visualization utilities

For more information: