VeloVAE (fullvb) benchmark on dyngen data#
Notebook benchmarks velocity and latent time inference using VeloVAE (fullvb) on dyngen-generated data.
Library imports#
import velovae as vv
import numpy as np
import pandas as pd
import torch
import anndata as ad
import scvelo as scv
from rgv_tools import DATA_DIR
from rgv_tools.benchmarking import get_velocity_correlation
/home/icb/yifan.chen/miniconda3/envs/regvelo-py310-velovae/lib/python3.10/site-packages/velovae/model/model_util.py:12: TqdmExperimentalWarning: Using `tqdm.autonotebook.tqdm` in notebook mode. Use `tqdm.tqdm` instead to force console mode (e.g. in jupyter console)
from tqdm.autonotebook import trange
General settings#
scv.settings.verbosity = 3
COMPLEXITY = "complexity_1"
Constants#
torch.manual_seed(0)
np.random.seed(0)
DATASET = "dyngen"
SAVE_DATA = True
if SAVE_DATA:
(DATA_DIR / DATASET / COMPLEXITY / "results").mkdir(parents=True, exist_ok=True)
(DATA_DIR / DATASET / COMPLEXITY / "processed" / "velovae_fullvb_vae").mkdir(parents=True, exist_ok=True)
SAVE_DATASETS = True
if SAVE_DATASETS:
(DATA_DIR / DATASET / COMPLEXITY / "trained_velovae_fullvb").mkdir(parents=True, exist_ok=True)
Velocity pipeline#
velocity_correlation = []
cnt = 0
for filename in (DATA_DIR / DATASET / COMPLEXITY / "processed").iterdir():
if filename.suffix != ".zarr":
continue
print(f"Run {cnt}, file {filename}.")
adata = ad.io.read_zarr(filename)
try:
rate_prior = {"alpha": (0.0, 1.0), "beta": (0.0, 0.5), "gamma": (0.0, 0.5)}
vae = vv.VAE(adata, tmax=20, dim_z=5, device="cuda:0", full_vb=True, rate_prior=rate_prior)
config = {}
vae.train(adata, config=config, plot=False, embed="pca")
simulation_id = int(filename.stem.removeprefix("simulation_"))
# Output velocity to adata object
vae.save_anndata(
adata,
"fullvb",
DATA_DIR / DATASET / COMPLEXITY / "processed" / "velovae_fullvb_vae",
file_name=f"velovae_fullvb_{simulation_id}.h5ad",
)
adata.layers["velocity"] = adata.layers["fullvb_velocity"].copy()
# save data
adata.write_zarr(DATA_DIR / DATASET / COMPLEXITY / "trained_velovae_fullvb" / f"trained_{simulation_id}.zarr")
velocity_correlation.append(
get_velocity_correlation(
ground_truth=adata.layers["true_velocity"], estimated=adata.layers["velocity"], aggregation=np.mean
)
)
except Exception as e: # noqa: BLE001
# Append np.nan in case of an error and optionally log the error
print(f"An error occurred: {e}")
velocity_correlation.append(np.nan)
cnt += 1
Run 0, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_29.zarr.
Estimating ODE parameters...
Detected 347 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
KS-test result: [1. 1. 1.]
Assign cluster 1 to repressive
Initial induction: 245, repression: 117/362
Learning Rate based on Data Sparsity: 0.0003
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 1: Early Stop Triggered at epoch 314. *********
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.147
Average Set Size: 21
********* Round 1: Early Stop Triggered at epoch 530. *********
Change in noise variance: 0.0336
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 579. *********
Change in noise variance: 0.0112
Change in x0: 0.4009
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 605. *********
Change in noise variance: 0.0027
Change in x0: 0.2125
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 631. *********
Change in noise variance: 0.0019
Change in x0: 0.1521
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 638. *********
Change in noise variance: 0.0005
Change in x0: 0.1149
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 669. *********
Change in noise variance: 0.0000
Change in x0: 0.1100
********* Velocity Refinement Round 7 *********
Stage 2: Early Stop Triggered at round 6.
********* Finished. Total Time = 0 h : 1 m : 14 s *********
Final: Train ELBO = 449.825, Test ELBO = 438.798
Run 1, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_14.zarr.
Estimating ODE parameters...
Detected 594 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.48, 0.8228239539949498), (0.52, 0.3684353137855576)
KS-test result: [0. 1. 1.]
Initial induction: 484, repression: 180/664
Learning Rate based on Data Sparsity: 0.0001
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 1: Early Stop Triggered at epoch 219. *********
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.034
Average Set Size: 49
********* Round 1: Early Stop Triggered at epoch 611. *********
Change in noise variance: 0.1225
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 667. *********
Change in noise variance: 0.0119
Change in x0: 1.1951
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 731. *********
Change in noise variance: 0.0223
Change in x0: 0.4867
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 765. *********
Change in noise variance: 0.0057
Change in x0: 0.2543
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 792. *********
Change in noise variance: 0.0003
Change in x0: 0.1477
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 843. *********
Change in noise variance: 0.0000
Change in x0: 0.1231
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 885. *********
Change in noise variance: 0.0000
Change in x0: 0.1125
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 904. *********
Change in noise variance: 0.0000
Change in x0: 0.0999
********* Velocity Refinement Round 9 *********
********* Round 9: Early Stop Triggered at epoch 951. *********
Change in noise variance: 0.0000
Change in x0: 0.0826
********* Velocity Refinement Round 10 *********
********* Round 10: Early Stop Triggered at epoch 1020. *********
Change in noise variance: 0.0000
Change in x0: 0.0774
********* Velocity Refinement Round 11 *********
Stage 2: Early Stop Triggered at round 10.
********* Finished. Total Time = 0 h : 1 m : 21 s *********
Final: Train ELBO = -328.411, Test ELBO = -399.075
Run 2, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_24.zarr.
Estimating ODE parameters...
Detected 461 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.59, 0.7142859350405649), (0.41, 0.232624469824922)
(0.52, 0.8547149661103973), (0.48, 0.31938394948827814)
KS-test result: [0. 0. 1.]
Initial induction: 431, repression: 295/726
Learning Rate based on Data Sparsity: 0.0000
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 1: Early Stop Triggered at epoch 9. *********
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.027
Average Set Size: 20
********* Round 1: Early Stop Triggered at epoch 18. *********
Change in noise variance: 0.2489
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 27. *********
Change in noise variance: 0.0050
Change in x0: 0.7320
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 36. *********
Change in noise variance: 0.0006
Change in x0: 0.4852
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 45. *********
Change in noise variance: 0.0000
Change in x0: 0.4468
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 54. *********
Change in noise variance: 0.0000
Change in x0: 0.3849
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 63. *********
Change in noise variance: 0.0000
Change in x0: 0.3637
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 72. *********
Change in noise variance: 0.0000
Change in x0: 0.3499
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 81. *********
Change in noise variance: 0.0000
Change in x0: 0.3190
********* Velocity Refinement Round 9 *********
********* Round 9: Early Stop Triggered at epoch 90. *********
Change in noise variance: 0.0000
Change in x0: 0.2829
********* Velocity Refinement Round 10 *********
********* Round 10: Early Stop Triggered at epoch 99. *********
Change in noise variance: 0.0000
Change in x0: 0.2803
********* Velocity Refinement Round 11 *********
Stage 2: Early Stop Triggered at round 10.
********* Finished. Total Time = 0 h : 0 m : 11 s *********
Final: Train ELBO = -26919.023, Test ELBO = -26492.256
Run 3, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_28.zarr.
Estimating ODE parameters...
Detected 535 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.39, 0.3808448852305928), (0.61, 0.852512757686449)
(0.46, 0.3875805469813231), (0.54, 0.7668672821296293)
KS-test result: [0. 0. 1.]
Initial induction: 374, repression: 190/564
Learning Rate based on Data Sparsity: 0.0002
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.005
Average Set Size: 22
Change in noise variance: 0.0401
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1541. *********
Change in noise variance: 0.0371
Change in x0: 0.3644
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1573. *********
Change in noise variance: 0.0020
Change in x0: 0.2457
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1580. *********
Change in noise variance: 0.0007
Change in x0: 0.2023
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1587. *********
Change in noise variance: 0.0000
Change in x0: 0.1785
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1594. *********
Change in noise variance: 0.0000
Change in x0: 0.1505
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1631. *********
Change in noise variance: 0.0000
Change in x0: 0.1245
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 1642. *********
Change in noise variance: 0.0000
Change in x0: 0.1042
********* Velocity Refinement Round 9 *********
********* Round 9: Early Stop Triggered at epoch 1649. *********
Change in noise variance: 0.0000
Change in x0: 0.0983
********* Velocity Refinement Round 10 *********
Stage 2: Early Stop Triggered at round 9.
********* Finished. Total Time = 0 h : 2 m : 12 s *********
Final: Train ELBO = 468.052, Test ELBO = 368.735
Run 4, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_6.zarr.
Estimating ODE parameters...
Detected 537 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.62, 0.8793261681749559), (0.38, 0.26633114834984656)
(0.47, 0.7331318775651423), (0.53, 0.2511832966547218)
(0.47, 0.2994057734262461), (0.53, 0.8553113871195344)
KS-test result: [0. 0. 0.]
Initial induction: 404, repression: 343/747
Learning Rate based on Data Sparsity: 0.0000
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.039
Average Set Size: 20
********* Round 1: Early Stop Triggered at epoch 1332. *********
Change in noise variance: 0.1963
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1529. *********
Change in noise variance: 0.0139
Change in x0: 0.7732
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1565. *********
Change in noise variance: 0.0039
Change in x0: 0.5783
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1644. *********
Change in noise variance: 0.0039
Change in x0: 0.5012
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1668. *********
Change in noise variance: 0.0016
Change in x0: 0.3899
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1682. *********
Change in noise variance: 0.0012
Change in x0: 0.3248
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1704. *********
Change in noise variance: 0.0008
Change in x0: 0.3053
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 1725. *********
Change in noise variance: 0.0000
Change in x0: 0.2925
********* Velocity Refinement Round 9 *********
********* Round 9: Early Stop Triggered at epoch 1732. *********
Change in noise variance: 0.0000
Change in x0: 0.2847
********* Velocity Refinement Round 10 *********
Stage 2: Early Stop Triggered at round 9.
********* Finished. Total Time = 0 h : 2 m : 20 s *********
Final: Train ELBO = -1471.624, Test ELBO = -1560.258
Run 5, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_21.zarr.
Estimating ODE parameters...
Detected 398 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.49, 0.2987033192902373), (0.51, 0.7362158867554179)
KS-test result: [1. 1. 0.]
Initial induction: 334, repression: 127/461
Learning Rate based on Data Sparsity: 0.0001
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.120
Average Set Size: 21
********* Round 1: Early Stop Triggered at epoch 1484. *********
Change in noise variance: 0.0695
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1640. *********
Change in noise variance: 0.0120
Change in x0: 0.3772
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1656. *********
Change in noise variance: 0.0045
Change in x0: 0.3035
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1762. *********
Change in noise variance: 0.0031
Change in x0: 0.1700
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1821. *********
Change in noise variance: 0.0010
Change in x0: 0.1158
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1838. *********
Change in noise variance: 0.0000
Change in x0: 0.0891
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1864. *********
Change in noise variance: 0.0000
Change in x0: 0.0845
********* Velocity Refinement Round 8 *********
Stage 2: Early Stop Triggered at round 7.
********* Finished. Total Time = 0 h : 2 m : 25 s *********
Final: Train ELBO = 640.215, Test ELBO = 555.901
Run 6, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_15.zarr.
Estimating ODE parameters...
Detected 404 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.62, 0.8595902088131244), (0.38, 0.34409679076346833)
(0.53, 0.3502039273194398), (0.47, 0.8163727115188388)
KS-test result: [0. 1. 0.]
Initial induction: 300, repression: 144/444
Learning Rate based on Data Sparsity: 0.0002
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 1: Early Stop Triggered at epoch 96. *********
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.229
Average Set Size: 106
********* Round 1: Early Stop Triggered at epoch 468. *********
Change in noise variance: 0.0993
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 485. *********
Change in noise variance: 0.0213
Change in x0: 1.9274
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 516. *********
Change in noise variance: 0.0172
Change in x0: 0.6222
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 583. *********
Change in noise variance: 0.0014
Change in x0: 0.1822
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 619. *********
Change in noise variance: 0.0002
Change in x0: 0.1456
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 628. *********
Change in noise variance: 0.0000
Change in x0: 0.1414
********* Velocity Refinement Round 7 *********
Stage 2: Early Stop Triggered at round 6.
********* Finished. Total Time = 0 h : 0 m : 49 s *********
Final: Train ELBO = -298.772, Test ELBO = -338.227
Run 7, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_9.zarr.
Estimating ODE parameters...
Detected 335 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.48, 0.37222433101870206), (0.52, 0.7806380011489331)
KS-test result: [1. 0. 1.]
Initial induction: 263, repression: 97/360
Learning Rate based on Data Sparsity: 0.0003
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.122
Average Set Size: 18
********* Round 1: Early Stop Triggered at epoch 1284. *********
Change in noise variance: 0.0227
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1355. *********
Change in noise variance: 0.0034
Change in x0: 0.1830
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1424. *********
Change in noise variance: 0.0002
Change in x0: 0.0994
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1503. *********
Change in noise variance: 0.0000
Change in x0: 0.0784
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1515. *********
Change in noise variance: 0.0000
Change in x0: 0.0766
********* Velocity Refinement Round 6 *********
Stage 2: Early Stop Triggered at round 5.
********* Finished. Total Time = 0 h : 1 m : 58 s *********
Final: Train ELBO = 493.208, Test ELBO = 467.782
Run 8, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_12.zarr.
Estimating ODE parameters...
Detected 387 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.59, 0.8421974990559944), (0.41, 0.31546207430654916)
(0.58, 0.7390920693790617), (0.42, 0.1625623840425744)
(0.42, 0.2930600982138266), (0.58, 0.8332787740830282)
KS-test result: [0. 0. 0.]
Initial induction: 310, repression: 205/515
Learning Rate based on Data Sparsity: 0.0000
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 5, test iteration: 8
********* Stage 1: Early Stop Triggered at epoch 9. *********
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.030
Average Set Size: 17
********* Round 1: Early Stop Triggered at epoch 18. *********
Change in noise variance: 0.1949
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 27. *********
Change in noise variance: 0.0019
Change in x0: 0.6673
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 36. *********
Change in noise variance: 0.0007
Change in x0: 0.5220
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 45. *********
Change in noise variance: 0.0000
Change in x0: 0.4418
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 54. *********
Change in noise variance: 0.0000
Change in x0: 0.3789
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 63. *********
Change in noise variance: 0.0000
Change in x0: 0.3652
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 72. *********
Change in noise variance: 0.0000
Change in x0: 0.2934
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 81. *********
Change in noise variance: 0.0000
Change in x0: 0.2694
********* Velocity Refinement Round 9 *********
********* Round 9: Early Stop Triggered at epoch 90. *********
Change in noise variance: 0.0000
Change in x0: 0.2871
********* Velocity Refinement Round 10 *********
Stage 2: Early Stop Triggered at round 9.
********* Finished. Total Time = 0 h : 0 m : 8 s *********
Final: Train ELBO = -23497.715, Test ELBO = -24467.719
Run 9, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_19.zarr.
Estimating ODE parameters...
Detected 244 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.46, 0.3700403220994146), (0.54, 0.8355761602987493)
(0.60, 0.33766490672002036), (0.40, 0.7567183237756051)
KS-test result: [0. 1. 0.]
Initial induction: 164, repression: 102/266
Learning Rate based on Data Sparsity: 0.0002
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.014
Average Set Size: 21
********* Round 1: Early Stop Triggered at epoch 1206. *********
Change in noise variance: 0.0339
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1363. *********
Change in noise variance: 0.0081
Change in x0: 0.3134
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1370. *********
Change in noise variance: 0.0016
Change in x0: 0.2070
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1446. *********
Change in noise variance: 0.0008
Change in x0: 0.1656
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1453. *********
Change in noise variance: 0.0000
Change in x0: 0.1375
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1537. *********
Change in noise variance: 0.0000
Change in x0: 0.1147
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1544. *********
Change in noise variance: 0.0000
Change in x0: 0.1026
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 1593. *********
Change in noise variance: 0.0000
Change in x0: 0.0901
********* Velocity Refinement Round 9 *********
********* Round 9: Early Stop Triggered at epoch 1610. *********
Change in noise variance: 0.0000
Change in x0: 0.0793
********* Velocity Refinement Round 10 *********
********* Round 10: Early Stop Triggered at epoch 1629. *********
Change in noise variance: 0.0000
Change in x0: 0.0719
********* Velocity Refinement Round 11 *********
Stage 2: Early Stop Triggered at round 10.
********* Finished. Total Time = 0 h : 2 m : 4 s *********
Final: Train ELBO = 268.584, Test ELBO = 244.117
Run 10, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_4.zarr.
Estimating ODE parameters...
Detected 454 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.59, 0.8009445508520957), (0.41, 0.36772632850943016)
KS-test result: [1. 0. 1.]
Initial induction: 368, repression: 111/479
Learning Rate based on Data Sparsity: 0.0003
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 1: Early Stop Triggered at epoch 731. *********
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.142
Average Set Size: 18
********* Round 1: Early Stop Triggered at epoch 907. *********
Change in noise variance: 0.0464
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 973. *********
Change in noise variance: 0.0051
Change in x0: 0.2664
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1014. *********
Change in noise variance: 0.0007
Change in x0: 0.1456
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1025. *********
Change in noise variance: 0.0000
Change in x0: 0.1260
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1094. *********
Change in noise variance: 0.0000
Change in x0: 0.1118
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1165. *********
Change in noise variance: 0.0000
Change in x0: 0.1064
********* Velocity Refinement Round 7 *********
Stage 2: Early Stop Triggered at round 6.
********* Finished. Total Time = 0 h : 1 m : 33 s *********
Final: Train ELBO = 587.956, Test ELBO = 546.589
Run 11, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_13.zarr.
Estimating ODE parameters...
Detected 502 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
KS-test result: [1. 1. 1.]
Assign cluster 0 to repressive
Initial induction: 315, repression: 199/514
Learning Rate based on Data Sparsity: 0.0004
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 1: Early Stop Triggered at epoch 579. *********
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.049
Average Set Size: 20
********* Round 1: Early Stop Triggered at epoch 900. *********
Change in noise variance: 0.0355
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 962. *********
Change in noise variance: 0.0139
Change in x0: 0.4108
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 969. *********
Change in noise variance: 0.0025
Change in x0: 0.2712
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1051. *********
Change in noise variance: 0.0009
Change in x0: 0.2049
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1075. *********
Change in noise variance: 0.0000
Change in x0: 0.1712
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1106. *********
Change in noise variance: 0.0000
Change in x0: 0.1455
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1125. *********
Change in noise variance: 0.0000
Change in x0: 0.1236
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 1132. *********
Change in noise variance: 0.0000
Change in x0: 0.1053
********* Velocity Refinement Round 9 *********
********* Round 9: Early Stop Triggered at epoch 1151. *********
Change in noise variance: 0.0000
Change in x0: 0.0940
********* Velocity Refinement Round 10 *********
********* Round 10: Early Stop Triggered at epoch 1167. *********
Change in noise variance: 0.0000
Change in x0: 0.0862
********* Velocity Refinement Round 11 *********
Stage 2: Early Stop Triggered at round 10.
********* Finished. Total Time = 0 h : 1 m : 32 s *********
Final: Train ELBO = 571.016, Test ELBO = 484.392
Run 12, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_2.zarr.
Estimating ODE parameters...
Detected 424 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.63, 0.7924425318102936), (0.37, 0.25047287353531905)
(0.32, 0.39095042714182476), (0.68, 0.9377617670621354)
(0.51, 0.35416668477069335), (0.49, 0.8927600435275674)
KS-test result: [0. 0. 0.]
Initial induction: 431, repression: 259/690
Learning Rate based on Data Sparsity: 0.0000
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.026
Average Set Size: 20
********* Round 1: Early Stop Triggered at epoch 1012. *********
Change in noise variance: 0.2075
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1021. *********
Change in noise variance: 0.0034
Change in x0: 0.9044
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1030. *********
Change in noise variance: 0.0019
Change in x0: 0.6653
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1039. *********
Change in noise variance: 0.0003
Change in x0: 0.5567
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1046. *********
Change in noise variance: 0.0000
Change in x0: 0.4385
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1053. *********
Change in noise variance: 0.0000
Change in x0: 0.3859
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1060. *********
Change in noise variance: 0.0000
Change in x0: 0.3471
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 1067. *********
Change in noise variance: 0.0000
Change in x0: 0.3200
********* Velocity Refinement Round 9 *********
********* Round 9: Early Stop Triggered at epoch 1074. *********
Change in noise variance: 0.0000
Change in x0: 0.3100
********* Velocity Refinement Round 10 *********
Stage 2: Early Stop Triggered at round 9.
********* Finished. Total Time = 0 h : 1 m : 30 s *********
Final: Train ELBO = -1431.609, Test ELBO = -1512.579
Run 13, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_16.zarr.
Estimating ODE parameters...
Detected 481 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
KS-test result: [1. 1. 1.]
Assign cluster 1 to repressive
Initial induction: 325, repression: 211/536
Learning Rate based on Data Sparsity: 0.0002
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.078
Average Set Size: 20
Change in noise variance: 0.0685
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1549. *********
Change in noise variance: 0.0195
Change in x0: 0.2868
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1613. *********
Change in noise variance: 0.0019
Change in x0: 0.2189
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1700. *********
Change in noise variance: 0.0004
Change in x0: 0.1530
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1734. *********
Change in noise variance: 0.0000
Change in x0: 0.1245
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1780. *********
Change in noise variance: 0.0000
Change in x0: 0.1096
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1862. *********
Change in noise variance: 0.0000
Change in x0: 0.0926
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 1901. *********
Change in noise variance: 0.0000
Change in x0: 0.0744
********* Velocity Refinement Round 9 *********
********* Round 9: Early Stop Triggered at epoch 1922. *********
Change in noise variance: 0.0000
Change in x0: 0.0624
********* Velocity Refinement Round 10 *********
********* Round 10: Early Stop Triggered at epoch 2086. *********
Change in noise variance: 0.0000
Change in x0: 0.0577
********* Velocity Refinement Round 11 *********
Stage 2: Early Stop Triggered at round 10.
********* Finished. Total Time = 0 h : 2 m : 43 s *********
Final: Train ELBO = 732.020, Test ELBO = 631.934
Run 14, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_1.zarr.
Estimating ODE parameters...
Detected 345 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
KS-test result: [1. 1. 1.]
Assign cluster 1 to repressive
Initial induction: 232, repression: 136/368
Learning Rate based on Data Sparsity: 0.0004
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.029
Average Set Size: 20
********* Round 1: Early Stop Triggered at epoch 1422. *********
Change in noise variance: 0.0486
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1458. *********
Change in noise variance: 0.0091
Change in x0: 0.3043
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1482. *********
Change in noise variance: 0.0005
Change in x0: 0.1621
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1568. *********
Change in noise variance: 0.0000
Change in x0: 0.1108
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1575. *********
Change in noise variance: 0.0000
Change in x0: 0.0903
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1586. *********
Change in noise variance: 0.0000
Change in x0: 0.0703
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1610. *********
Change in noise variance: 0.0000
Change in x0: 0.0597
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 1621. *********
Change in noise variance: 0.0000
Change in x0: 0.0533
********* Velocity Refinement Round 9 *********
Stage 2: Early Stop Triggered at round 8.
********* Finished. Total Time = 0 h : 2 m : 7 s *********
Final: Train ELBO = 558.691, Test ELBO = 509.981
Run 15, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_18.zarr.
Estimating ODE parameters...
Detected 327 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.51, 0.3274513378440137), (0.49, 0.8157796920163066)
KS-test result: [1. 1. 0.]
Initial induction: 264, repression: 100/364
Learning Rate based on Data Sparsity: 0.0002
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.034
Average Set Size: 20
Change in noise variance: 0.0658
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1572. *********
Change in noise variance: 0.0105
Change in x0: 0.2625
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1616. *********
Change in noise variance: 0.0015
Change in x0: 0.1819
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1725. *********
Change in noise variance: 0.0004
Change in x0: 0.1387
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1816. *********
Change in noise variance: 0.0000
Change in x0: 0.1053
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1943. *********
Change in noise variance: 0.0000
Change in x0: 0.0764
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 2144. *********
Change in noise variance: 0.0000
Change in x0: 0.0618
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 2200. *********
Change in noise variance: 0.0000
Change in x0: 0.0553
********* Velocity Refinement Round 9 *********
Stage 2: Early Stop Triggered at round 8.
********* Finished. Total Time = 0 h : 2 m : 49 s *********
Final: Train ELBO = 699.190, Test ELBO = 641.937
Run 16, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_5.zarr.
Estimating ODE parameters...
Detected 262 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
KS-test result: [1. 1. 1.]
Assign cluster 1 to repressive
Initial induction: 182, repression: 100/282
Learning Rate based on Data Sparsity: 0.0003
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.141
Average Set Size: 18
********* Round 1: Early Stop Triggered at epoch 1274. *********
Change in noise variance: 0.0475
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1396. *********
Change in noise variance: 0.0169
Change in x0: 0.2664
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1455. *********
Change in noise variance: 0.0008
Change in x0: 0.1681
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1489. *********
Change in noise variance: 0.0000
Change in x0: 0.1470
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1496. *********
Change in noise variance: 0.0000
Change in x0: 0.1528
********* Velocity Refinement Round 6 *********
Stage 2: Early Stop Triggered at round 5.
********* Finished. Total Time = 0 h : 1 m : 57 s *********
Final: Train ELBO = 238.936, Test ELBO = 203.147
Run 17, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_10.zarr.
Estimating ODE parameters...
Detected 306 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.44, 0.21427183785803913), (0.56, 0.8053766111264617)
(0.65, 0.8997967816308998), (0.35, 0.35802780150367175)
KS-test result: [0. 1. 0.]
Initial induction: 379, repression: 169/548
Learning Rate based on Data Sparsity: 0.0000
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 1: Early Stop Triggered at epoch 9. *********
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.035
Average Set Size: 19
********* Round 1: Early Stop Triggered at epoch 30. *********
Change in noise variance: 0.2346
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 37. *********
Change in noise variance: 0.0073
Change in x0: 0.6161
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 44. *********
Change in noise variance: 0.0039
Change in x0: 0.5040
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 51. *********
Change in noise variance: 0.0018
Change in x0: 0.4250
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 60. *********
Change in noise variance: 0.0007
Change in x0: 0.3543
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 67. *********
Change in noise variance: 0.0000
Change in x0: 0.3236
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 74. *********
Change in noise variance: 0.0000
Change in x0: 0.2708
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 81. *********
Change in noise variance: 0.0000
Change in x0: 0.2704
********* Velocity Refinement Round 9 *********
Stage 2: Early Stop Triggered at round 8.
********* Finished. Total Time = 0 h : 0 m : 9 s *********
Final: Train ELBO = -24367.305, Test ELBO = -24067.287
Run 18, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_8.zarr.
Estimating ODE parameters...
Detected 397 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
KS-test result: [1. 1. 1.]
Assign cluster 0 to repressive
Initial induction: 295, repression: 167/462
Learning Rate based on Data Sparsity: 0.0003
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.024
Average Set Size: 20
********* Round 1: Early Stop Triggered at epoch 1431. *********
Change in noise variance: 0.0907
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1510. *********
Change in noise variance: 0.0053
Change in x0: 0.2621
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1557. *********
Change in noise variance: 0.0008
Change in x0: 0.1654
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1614. *********
Change in noise variance: 0.0000
Change in x0: 0.1322
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1665. *********
Change in noise variance: 0.0000
Change in x0: 0.1163
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1672. *********
Change in noise variance: 0.0000
Change in x0: 0.1057
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1734. *********
Change in noise variance: 0.0000
Change in x0: 0.0897
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 1801. *********
Change in noise variance: 0.0000
Change in x0: 0.0756
********* Velocity Refinement Round 9 *********
********* Round 9: Early Stop Triggered at epoch 1828. *********
Change in noise variance: 0.0000
Change in x0: 0.0694
********* Velocity Refinement Round 10 *********
Stage 2: Early Stop Triggered at round 9.
********* Finished. Total Time = 0 h : 2 m : 23 s *********
Final: Train ELBO = 892.707, Test ELBO = 736.356
Run 19, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_11.zarr.
Estimating ODE parameters...
Detected 346 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.59, 0.8267164296804612), (0.41, 0.36831379905588)
KS-test result: [1. 0. 1.]
Initial induction: 291, repression: 86/377
Learning Rate based on Data Sparsity: 0.0002
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.141
Average Set Size: 18
********* Round 1: Early Stop Triggered at epoch 1367. *********
Change in noise variance: 0.0236
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1409. *********
Change in noise variance: 0.0025
Change in x0: 0.1406
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1423. *********
Change in noise variance: 0.0006
Change in x0: 0.0880
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1472. *********
Change in noise variance: 0.0000
Change in x0: 0.0785
********* Velocity Refinement Round 5 *********
Stage 2: Early Stop Triggered at round 4.
********* Finished. Total Time = 0 h : 1 m : 55 s *********
Final: Train ELBO = 504.837, Test ELBO = 472.133
Run 20, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_27.zarr.
Estimating ODE parameters...
Detected 320 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.39, 0.28728312975125536), (0.61, 0.8005818061651941)
KS-test result: [0. 1. 1.]
Initial induction: 289, repression: 92/381
Learning Rate based on Data Sparsity: 0.0002
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.019
Average Set Size: 21
Change in noise variance: 0.0556
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1544. *********
Change in noise variance: 0.0453
Change in x0: 0.3729
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1611. *********
Change in noise variance: 0.0041
Change in x0: 0.2411
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1618. *********
Change in noise variance: 0.0014
Change in x0: 0.1940
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1625. *********
Change in noise variance: 0.0005
Change in x0: 0.1519
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1644. *********
Change in noise variance: 0.0000
Change in x0: 0.1335
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1655. *********
Change in noise variance: 0.0000
Change in x0: 0.1172
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 1664. *********
Change in noise variance: 0.0000
Change in x0: 0.1075
********* Velocity Refinement Round 9 *********
Stage 2: Early Stop Triggered at round 8.
********* Finished. Total Time = 0 h : 2 m : 9 s *********
Final: Train ELBO = 331.373, Test ELBO = 255.813
Run 21, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_23.zarr.
Estimating ODE parameters...
Detected 326 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.51, 0.7911187894111076), (0.49, 0.37965464284669165)
KS-test result: [1. 1. 0.]
Initial induction: 264, repression: 95/359
Learning Rate based on Data Sparsity: 0.0003
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 1: Early Stop Triggered at epoch 754. *********
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.144
Average Set Size: 19
********* Round 1: Early Stop Triggered at epoch 921. *********
Change in noise variance: 0.0380
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 955. *********
Change in noise variance: 0.0062
Change in x0: 0.2466
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 991. *********
Change in noise variance: 0.0002
Change in x0: 0.1567
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 998. *********
Change in noise variance: 0.0000
Change in x0: 0.1274
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1010. *********
Change in noise variance: 0.0000
Change in x0: 0.1248
********* Velocity Refinement Round 6 *********
Stage 2: Early Stop Triggered at round 5.
********* Finished. Total Time = 0 h : 1 m : 20 s *********
Final: Train ELBO = 486.095, Test ELBO = 463.734
Run 22, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_17.zarr.
Estimating ODE parameters...
Detected 410 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
KS-test result: [1. 1. 1.]
Assign cluster 1 to repressive
Initial induction: 284, repression: 166/450
Learning Rate based on Data Sparsity: 0.0004
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.053
Average Set Size: 23
********* Round 1: Early Stop Triggered at epoch 1434. *********
Change in noise variance: 0.0677
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1681. *********
Change in noise variance: 0.0075
Change in x0: 0.2493
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1787. *********
Change in noise variance: 0.0007
Change in x0: 0.1443
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1848. *********
Change in noise variance: 0.0000
Change in x0: 0.1204
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1955. *********
Change in noise variance: 0.0000
Change in x0: 0.0924
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 2036. *********
Change in noise variance: 0.0000
Change in x0: 0.0691
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 2213. *********
Change in noise variance: 0.0000
Change in x0: 0.0653
********* Velocity Refinement Round 8 *********
Stage 2: Early Stop Triggered at round 7.
********* Finished. Total Time = 0 h : 2 m : 51 s *********
Final: Train ELBO = 853.947, Test ELBO = 717.942
Run 23, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_30.zarr.
Estimating ODE parameters...
Detected 404 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.64, 0.7874540650534966), (0.36, 0.30795453403738815)
(0.67, 0.8559480355355736), (0.33, 0.3295921295742867)
KS-test result: [0. 0. 1.]
Initial induction: 337, repression: 128/465
Learning Rate based on Data Sparsity: 0.0000
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.029
Average Set Size: 20
********* Round 1: Early Stop Triggered at epoch 1271. *********
Change in noise variance: 0.0555
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1280. *********
Change in noise variance: 0.0262
Change in x0: 0.3258
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1312. *********
Change in noise variance: 0.0027
Change in x0: 0.2685
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1338. *********
Change in noise variance: 0.0012
Change in x0: 0.2391
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1362. *********
Change in noise variance: 0.0015
Change in x0: 0.2168
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1413. *********
Change in noise variance: 0.0014
Change in x0: 0.1821
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1430. *********
Change in noise variance: 0.0011
Change in x0: 0.1758
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 1447. *********
Change in noise variance: 0.0011
Change in x0: 0.1639
********* Velocity Refinement Round 9 *********
********* Round 9: Early Stop Triggered at epoch 1469. *********
Change in noise variance: 0.0006
Change in x0: 0.1284
********* Velocity Refinement Round 10 *********
********* Round 10: Early Stop Triggered at epoch 1476. *********
Change in noise variance: 0.0000
Change in x0: 0.1102
********* Velocity Refinement Round 11 *********
********* Round 11: Early Stop Triggered at epoch 1483. *********
Change in noise variance: 0.0000
Change in x0: 0.0967
********* Velocity Refinement Round 12 *********
********* Round 12: Early Stop Triggered at epoch 1499. *********
Change in noise variance: 0.0000
Change in x0: 0.0821
********* Velocity Refinement Round 13 *********
********* Round 13: Early Stop Triggered at epoch 1548. *********
Change in noise variance: 0.0000
Change in x0: 0.0643
********* Velocity Refinement Round 14 *********
********* Round 14: Early Stop Triggered at epoch 1557. *********
Change in noise variance: 0.0000
Change in x0: 0.0579
********* Velocity Refinement Round 15 *********
Stage 2: Early Stop Triggered at round 14.
********* Finished. Total Time = 0 h : 2 m : 4 s *********
Final: Train ELBO = -2.364, Test ELBO = -59.837
Run 24, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_22.zarr.
Estimating ODE parameters...
Detected 393 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.66, 0.7192424805256069), (0.34, 0.29799031256004654)
KS-test result: [0. 1. 1.]
Initial induction: 333, repression: 89/422
Learning Rate based on Data Sparsity: 0.0004
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 1: Early Stop Triggered at epoch 989. *********
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.019
Average Set Size: 19
********* Round 1: Early Stop Triggered at epoch 1308. *********
Change in noise variance: 0.0429
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1367. *********
Change in noise variance: 0.0066
Change in x0: 0.2664
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1393. *********
Change in noise variance: 0.0024
Change in x0: 0.1964
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1497. *********
Change in noise variance: 0.0011
Change in x0: 0.1703
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1589. *********
Change in noise variance: 0.0008
Change in x0: 0.1515
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1655. *********
Change in noise variance: 0.0000
Change in x0: 0.1359
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1699. *********
Change in noise variance: 0.0000
Change in x0: 0.1163
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 1813. *********
Change in noise variance: 0.0000
Change in x0: 0.1072
********* Velocity Refinement Round 9 *********
Stage 2: Early Stop Triggered at round 8.
********* Finished. Total Time = 0 h : 2 m : 21 s *********
Final: Train ELBO = 616.538, Test ELBO = 526.494
Run 25, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_25.zarr.
Estimating ODE parameters...
Detected 453 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.52, 0.28603679504748397), (0.48, 0.7673497264953757)
(0.37, 0.33761410289326726), (0.63, 0.8504461451142085)
KS-test result: [0. 1. 0.]
Initial induction: 369, repression: 209/578
Learning Rate based on Data Sparsity: 0.0000
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.011
Average Set Size: 46
Change in noise variance: 0.2165
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1581. *********
Change in noise variance: 0.0129
Change in x0: 0.9887
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1718. *********
Change in noise variance: 0.0076
Change in x0: 0.7500
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1729. *********
Change in noise variance: 0.0109
Change in x0: 0.6287
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1766. *********
Change in noise variance: 0.0053
Change in x0: 0.5312
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1773. *********
Change in noise variance: 0.0021
Change in x0: 0.4532
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1797. *********
Change in noise variance: 0.0049
Change in x0: 0.3519
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 1828. *********
Change in noise variance: 0.0057
Change in x0: 0.2352
********* Velocity Refinement Round 9 *********
********* Round 9: Early Stop Triggered at epoch 1854. *********
Change in noise variance: 0.0038
Change in x0: 0.1470
********* Velocity Refinement Round 10 *********
********* Round 10: Early Stop Triggered at epoch 1896. *********
Change in noise variance: 0.0015
Change in x0: 0.1019
********* Velocity Refinement Round 11 *********
********* Round 11: Early Stop Triggered at epoch 1907. *********
Change in noise variance: 0.0004
Change in x0: 0.0837
********* Velocity Refinement Round 12 *********
********* Round 12: Early Stop Triggered at epoch 1946. *********
Change in noise variance: 0.0000
Change in x0: 0.0917
********* Velocity Refinement Round 13 *********
Stage 2: Early Stop Triggered at round 12.
********* Finished. Total Time = 0 h : 2 m : 36 s *********
Final: Train ELBO = -1092.304, Test ELBO = -1252.312
Run 26, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_20.zarr.
Estimating ODE parameters...
Detected 280 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
KS-test result: [1. 1. 1.]
Assign cluster 2 to repressive
Initial induction: 201, repression: 88/289
Learning Rate based on Data Sparsity: 0.0004
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.118
Average Set Size: 19
********* Round 1: Early Stop Triggered at epoch 1452. *********
Change in noise variance: 0.0595
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1629. *********
Change in noise variance: 0.0121
Change in x0: 0.2074
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1705. *********
Change in noise variance: 0.0010
Change in x0: 0.1123
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1817. *********
Change in noise variance: 0.0002
Change in x0: 0.0804
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1986. *********
Change in noise variance: 0.0000
Change in x0: 0.0608
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 2155. *********
Change in noise variance: 0.0000
Change in x0: 0.0600
********* Velocity Refinement Round 7 *********
Stage 2: Early Stop Triggered at round 6.
********* Finished. Total Time = 0 h : 2 m : 44 s *********
Final: Train ELBO = 523.419, Test ELBO = 410.966
Run 27, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_7.zarr.
Estimating ODE parameters...
Detected 347 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.35, 0.3338688963765141), (0.65, 0.8501683610682383)
(0.43, 0.26033194504377993), (0.57, 0.7905961327414116)
KS-test result: [0. 1. 0.]
Initial induction: 340, repression: 141/481
Learning Rate based on Data Sparsity: 0.0000
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.033
Average Set Size: 20
********* Round 1: Early Stop Triggered at epoch 1416. *********
Change in noise variance: 0.0589
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1478. *********
Change in noise variance: 0.0218
Change in x0: 0.4051
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1590. *********
Change in noise variance: 0.0033
Change in x0: 0.3137
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1666. *********
Change in noise variance: 0.0026
Change in x0: 0.2701
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1695. *********
Change in noise variance: 0.0013
Change in x0: 0.2180
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1762. *********
Change in noise variance: 0.0009
Change in x0: 0.1583
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1769. *********
Change in noise variance: 0.0000
Change in x0: 0.1418
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 1776. *********
Change in noise variance: 0.0000
Change in x0: 0.1210
********* Velocity Refinement Round 9 *********
********* Round 9: Early Stop Triggered at epoch 1828. *********
Change in noise variance: 0.0000
Change in x0: 0.1092
********* Velocity Refinement Round 10 *********
********* Round 10: Early Stop Triggered at epoch 1902. *********
Change in noise variance: 0.0000
Change in x0: 0.1064
********* Velocity Refinement Round 11 *********
Stage 2: Early Stop Triggered at round 10.
********* Finished. Total Time = 0 h : 2 m : 29 s *********
Final: Train ELBO = -124.616, Test ELBO = -153.157
Run 28, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_3.zarr.
Estimating ODE parameters...
Detected 148 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
(0.41, 0.7019526186807664), (0.59, 0.16143693643502346)
KS-test result: [1. 0. 1.]
Initial induction: 137, repression: 71/208
Learning Rate based on Data Sparsity: 0.0000
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 1: Early Stop Triggered at epoch 9. *********
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.030
Average Set Size: 20
********* Round 1: Early Stop Triggered at epoch 18. *********
Change in noise variance: 0.2526
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 25. *********
Change in noise variance: 0.0175
Change in x0: 0.5068
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 34. *********
Change in noise variance: 0.0136
Change in x0: 0.3108
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 43. *********
Change in noise variance: 0.0037
Change in x0: 0.1939
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 52. *********
Change in noise variance: 0.0009
Change in x0: 0.1428
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 59. *********
Change in noise variance: 0.0000
Change in x0: 0.1115
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 73. *********
Change in noise variance: 0.0000
Change in x0: 0.1003
********* Velocity Refinement Round 8 *********
********* Round 8: Early Stop Triggered at epoch 82. *********
Change in noise variance: 0.0000
Change in x0: 0.0981
********* Velocity Refinement Round 9 *********
Stage 2: Early Stop Triggered at round 8.
********* Finished. Total Time = 0 h : 0 m : 7 s *********
Final: Train ELBO = -37723.707, Test ELBO = -37839.289
Run 29, file /ictstr01/groups/ml01/workspace/yifan.chen/regvelo_reproducibility/data/dyngen/complexity_1/processed/simulation_26.zarr.
Estimating ODE parameters...
Detected 470 velocity genes.
Estimating the variance...
Initialization using the steady-state and dynamical models.
Reinitialize the regular ODE parameters based on estimated global latent time.
3 clusters detected based on gene co-expression.
KS-test result: [1. 1. 1.]
Assign cluster 1 to repressive
Initial induction: 324, repression: 170/494
Learning Rate based on Data Sparsity: 0.0003
--------------------------- Train a VeloVAE ---------------------------
********* Creating Training/Validation Datasets *********
********* Finished. *********
********* Creating optimizers *********
********* Finished. *********
********* Start training *********
********* Stage 1 *********
Total Number of Iterations Per Epoch: 6, test iteration: 10
********* Stage 1: Early Stop Triggered at epoch 824. *********
********* Stage 2 *********
********* Velocity Refinement Round 1 *********
Percentage of Invalid Sets: 0.118
Average Set Size: 19
********* Round 1: Early Stop Triggered at epoch 1041. *********
Change in noise variance: 0.0273
********* Velocity Refinement Round 2 *********
********* Round 2: Early Stop Triggered at epoch 1113. *********
Change in noise variance: 0.0124
Change in x0: 0.2851
********* Velocity Refinement Round 3 *********
********* Round 3: Early Stop Triggered at epoch 1132. *********
Change in noise variance: 0.0018
Change in x0: 0.1825
********* Velocity Refinement Round 4 *********
********* Round 4: Early Stop Triggered at epoch 1156. *********
Change in noise variance: 0.0004
Change in x0: 0.1316
********* Velocity Refinement Round 5 *********
********* Round 5: Early Stop Triggered at epoch 1182. *********
Change in noise variance: 0.0000
Change in x0: 0.1173
********* Velocity Refinement Round 6 *********
********* Round 6: Early Stop Triggered at epoch 1204. *********
Change in noise variance: 0.0000
Change in x0: 0.0974
********* Velocity Refinement Round 7 *********
********* Round 7: Early Stop Triggered at epoch 1228. *********
Change in noise variance: 0.0000
Change in x0: 0.0896
********* Velocity Refinement Round 8 *********
Stage 2: Early Stop Triggered at round 7.
********* Finished. Total Time = 0 h : 1 m : 38 s *********
Final: Train ELBO = 594.783, Test ELBO = 521.406
Data saving#
if SAVE_DATA:
pd.DataFrame({"velocity": velocity_correlation}).to_parquet(
path=DATA_DIR / DATASET / COMPLEXITY / "results" / "velovae_fullvb_correlation.parquet"
)