Research Archive

Journal Article Open Access Bioinformatics

Single-Cell RNA Sequencing Data Analysis Pipelines: Scalable Dimensionality Reduction, Cell Type Clustering, and Trajectory Inference for Million-Cell Atlas Construction

Single-cell RNA sequencing (scRNA-seq) has transformed cell biology by enabling genome-wide transcriptomic profiling at single-cell resolution, but the computational pipelines required to process, normalize, cluster, and interpret datasets at the scale of million-cell atlases demand engineering solutions that go substantially beyond the academic prototype tools in common use. This paper presents ScaleSC, a horizontally scalable scRNA-seq analysis pipeline designed for cloud cluster deployment, and evaluates its performance on datasets ranging from 10,000 to 4.2 million cells against the Seurat, Scanpy, and RAPIDS cuML pipelines. ScaleSC implements distributed PCA using a randomized SVD algorithm that scales linearly with cell count on Apache Spark clusters, a GPU-accelerated UMAP implementation achieving 18x speedup over CPU UMAP at one million cells, and a graph-based clustering module supporting both Leiden and Louvain algorithms with adaptive resolution selection. On the 4.2-million-cell Human Cell Atlas bone marrow dataset, ScaleSC completes the full analysis pipeline in 47 minutes on a 32-node cluster, compared to 14.3 hours for Scanpy on the same hardware. Cell type assignment accuracy (benchmarked against expert-annotated ground truth labels) is 94.2 percent using ScaleSC marker-gene transfer, versus 91.8 percent for Seurat v4 label transfer. We release ScaleSC as open-source software with Docker-based deployment pipelines and cloud infrastructure templates for AWS, GCP, and Azure.

Chukwuebuka Aneke, Astrid Karlsson, Masashi Yamada, Amira El-Sayed· Jul 2020· 389 citations

Journal Article Open Access Artificial Intelligence

Machine Learning-Augmented DevOps: Automated Anomaly Detection and Predictive Incident Management in High-Velocity Deployment Environments

The increasing velocity of software deployments enabled by mature CI/CD practices has outpaced the capacity of human operators to detect and respond to production incidents through manual monitoring. This paper explores the integration of machine learning techniques into DevOps operational pipelines — an emerging discipline termed AIOps — with particular focus on anomaly detection and predictive incident management. We present ML-DevOps, a reference architecture that integrates unsupervised anomaly detection models (Isolation Forest, LSTM Autoencoders) with supervised incident classifiers into a continuous delivery pipeline. The architecture is evaluated using a real-world dataset comprising 14 months of telemetry from a large e-commerce platform processing over 2 million daily transactions. ML-DevOps achieves a 91.3% anomaly detection precision and a 5.2-minute mean advance warning time before customer-impacting incidents, representing an 82% improvement over threshold-based alerting baselines. We further analyze model drift in the context of continuous deployment, demonstrating that retraining frequency must scale with deployment frequency to maintain detection accuracy. This work bridges the gap between machine learning research and DevOps practice, providing both an architectural blueprint and empirical evidence for AIOps integration.

Victoria Osei, Daniel Reinhardt, Yuki Tanaka, Fatima Al-Rashidi· Apr 2020· 589 citations

Journal Article Subscription Cybersecurity

Zero Trust Architecture Implementation in Enterprise Networks: A Comparative Study of Identity-Centric, Microsegmentation, and Software-Defined Perimeter Approaches

Zero Trust Architecture (ZTA) has emerged as the prevailing framework for enterprise network security in the aftermath of high-profile perimeter breach incidents, premised on the principle that no user, device, or network segment should be implicitly trusted regardless of its position relative to the network boundary. Despite widespread adoption of ZTA terminology, organizational implementations vary dramatically in scope, completeness, and operational effectiveness. This paper presents a comparative analysis of three primary ZTA implementation approaches -- Identity-Centric ZTA (exemplified by Google BeyondCorp), Microsegmentation-Based ZTA (exemplified by Illumio and Guardicore), and Software-Defined Perimeter ZTA (exemplified by Zscaler and Cloudflare Access) -- across eight enterprise deployments. Using a standardized ZTA Maturity Assessment Instrument (ZTMAI) comprising 47 control dimensions mapped to NIST SP 800-207, we evaluate each deployment against access policy granularity, continuous authentication coverage, lateral movement containment, data exfiltration prevention effectiveness, and operational complexity. Identity-Centric implementations achieve the highest ZTMAI scores in continuous authentication and data access governance dimensions but require substantial identity infrastructure investment. Microsegmentation delivers the strongest lateral movement containment (mean blast radius reduction of 91%) but incurs the highest policy management overhead. We provide a ZTA implementation selection framework based on organizational threat model, existing infrastructure, and engineering capacity.

Obiageli Chukwu, Malin Eriksson, Takuya Yamada, Amira El-Sayed· Apr 2020· 452 citations

Journal Article Open Access Quantum Computing

Quantum Error Correction Codes for Noisy Intermediate-Scale Quantum Devices: Implementation Trade-offs of Surface Codes, Steane Codes, and Bacon-Shor Codes on Superconducting Qubit Architectures

Quantum error correction (QEC) is widely recognized as a prerequisite for fault-tolerant quantum computation, yet the overhead requirements of leading QEC codes -- in terms of physical-to-logical qubit ratios and gate operation counts -- exceed the capabilities of current Noisy Intermediate-Scale Quantum (NISQ) devices by orders of magnitude. This paper investigates the implementation trade-offs of three QEC code families -- Surface Codes, Steane Codes, and Bacon-Shor Codes -- on superconducting transmon qubit architectures representative of current IBM Quantum and Google Sycamore hardware generations. Using a hardware-calibrated noise model derived from publicly available device characterization data, we simulate QEC circuit performance across logical qubit distances 3 through 9, measuring logical error rate suppression, syndrome extraction circuit depth, connectivity requirements, and decoding latency. Surface codes achieve the best logical error rate suppression per physical qubit overhead at distance 5 (logical error rate 2.3x10-4 at 0.1% physical gate error rate), but require all-nearest-neighbor connectivity that strains current device topologies. Bacon-Shor codes demonstrate the lowest syndrome extraction circuit depth, making them favorable for architectures with limited two-qubit gate fidelity. We introduce a QEC Code Suitability Index (QCSI) that maps device connectivity, gate fidelity, and coherence time profiles to code family recommendations, and apply it across six current quantum hardware platforms.

Chukwuemeka Obialo, Astrid Lindqvist, Hiroshi Nishimura, Rania Ahmed· Jan 2020· 387 citations

Journal Article Open Access Software Engineering

Shift-Left Performance Engineering: Integrating Load Testing, Profiling, and Performance Budgets into DevOps Pipelines

Performance degradation in production software systems frequently originates in code changes that pass functional testing but introduce latency regressions, memory leaks, or throughput reductions that only manifest at scale. Shift-left performance engineering addresses this through early, continuous performance validation integrated into CI/CD pipelines. This paper presents a comprehensive framework for shift-left performance engineering, grounded in a practitioner survey (n=319) and a longitudinal case study of four organizations that transitioned from periodic load testing to continuous pipeline-integrated performance validation. We define the Performance Gate Model, comprising three gate types — microbenchmark gates at unit level, service-level load test gates at integration level, and synthetic traffic replay gates at staging level — and demonstrate their complementary fault detection profiles. Our longitudinal analysis shows that organizations implementing all three gate types detect 89% of performance regressions before production deployment, compared to 31% for organizations relying solely on post-deployment monitoring. We evaluate toolchain options including k6, Gatling, Apache JMeter, and Locust for pipeline integration, comparing their CI/CD ergonomics, scripting model, and result visualization capabilities. Performance budget enforcement — defining and rejecting builds that violate response time, error rate, or throughput thresholds — is identified as the highest-leverage single practice, adopted by only 24% of surveyed organizations despite its measurable impact.

Olawale Adeyemi, Kristina Magnusson, Ryuichi Yamada, Inês Carvalho· Jan 2020· 334 citations

Journal Article Open Access Green Computing

Energy Proportionality in Large-Scale Data Centers: Power Usage Effectiveness Optimization Through Workload Consolidation, Cooling System Intelligence, and Renewable Energy Integration

Data centers globally consume approximately 1-2 percent of world electricity supply, with power usage effectiveness (PUE) -- the ratio of total facility power to IT equipment power -- serving as the primary industry efficiency benchmark. Despite significant progress in server-level energy proportionality, facility-level PUE optimization through the joint management of workload consolidation, cooling system intelligence, and renewable energy procurement presents substantial unresolved engineering challenges. This paper presents DataGreen, an integrated energy management framework for large-scale data centers that coordinates workload scheduling, cooling control, and renewable energy utilization through a model predictive control architecture. DataGreen is evaluated through a combination of simulation using a validated data center energy model and a 12-month deployment in a 15 MW hyperscale data center. DataGreen achieves a mean annual PUE of 1.12 compared to the baseline PUE of 1.34, representing a 16.4 percent reduction in total facility power. Workload consolidation using thermal-aware VM placement contributes a 7.1 percent power reduction, DeepMind-inspired ML-based cooling optimization contributes 6.8 percent, and renewable energy time-shifting (scheduling deferrable workloads to periods of high renewable availability) contributes an effective 9.2 percent reduction in carbon intensity. We introduce the Data Center Carbon Efficiency Index (DCCEI) that combines PUE with renewable energy fraction and workload carbon intensity, and demonstrate that DCCEI provides a more complete sustainability picture than PUE alone.

Seun Adesanya, Hanna Bergstrom, Ryo Kawamoto, Nadia Khalil· Oct 2019· 312 citations

Journal Article Subscription Software Engineering

Database DevOps: Version-Controlled Schema Migration, Automated Database Testing, and Continuous Delivery of Persistent Data Layer Changes

Despite significant maturation of application-layer continuous delivery practices, the database tier remains one of the most common manual intervention points in otherwise automated DevOps pipelines. Schema migrations, data migrations, and stored procedure deployments are frequently managed through ad-hoc scripts and manual DBA approval gates that fragment delivery pipelines and introduce deployment risk. This paper presents Database DevOps as a coherent engineering discipline addressing the full lifecycle of database change management within CI/CD workflows. We evaluate three schema migration frameworks — Flyway, Liquibase, and Alembic — against six operational criteria: migration idempotency, rollback capability, branching support, CI integration maturity, drift detection, and cross-engine portability. We further characterize a taxonomy of database testing strategies — schema contract tests, referential integrity tests, performance regression tests, and data quality invariant tests — and provide empirical evidence of their fault detection effectiveness from a controlled experiment involving 1,200 intentionally injected database defects. Organizations implementing full Database DevOps practices in our case studies reduced database-related deployment failures by 71% and eliminated manual DBA gates in 83% of deployment pathways. We provide a Database DevOps Implementation Roadmap as a practitioner artifact.

Adunola Fashola, Erik Johansson, Daisuke Mori, Ana Sofia Ferreira· Oct 2019· 256 citations

Journal Article Open Access Cloud Computing

Federated Learning in Distributed Cloud Environments: Communication Efficiency, Differential Privacy Guarantees, and Model Convergence Under Non-IID Data Distributions

Federated learning enables collaborative model training across distributed clients without centralizing raw data, making it particularly valuable in regulated industries where data sharing is legally constrained. However, the practical deployment of federated learning in production cloud environments exposes significant engineering challenges: communication overhead from model update aggregation, degraded convergence under non-IID (heterogeneous) data distributions, and the tension between differential privacy noise injection and model utility. This paper presents FedCloud, a production-oriented federated learning framework designed for multi-organization deployment across cloud environments, and reports its empirical evaluation across three deployment scenarios: cross-hospital clinical NLP, cross-bank fraud detection, and cross-retailer demand forecasting. FedCloud implements adaptive FedAvg with gradient compression (achieving 7.3x communication reduction), client selection strategies optimized for stragglers in cloud environments, and Renyi differential privacy with per-round epsilon tracking. Under non-IID distributions simulating real organizational data heterogeneity, FedCloud achieves within 2.8% of centralized training accuracy for the clinical NLP task and within 4.1% for fraud detection, while satisfying epsilon less than 3 per training round. We characterize the accuracy-privacy-communication three-way trade-off surface empirically and provide a configuration selection guide for practitioners deploying federated learning under specific regulatory and performance constraints.

Nkechi Eze, Lars Berggren, Akira Yamamoto, Nadia Khalil· Jul 2019· 441 citations

Journal Article Subscription Cloud Computing

Serverless Computing in DevOps Pipelines: Performance Trade-offs, Cold Start Mitigation, and Organizational Adoption Patterns

Serverless computing platforms have disrupted traditional DevOps deployment models by abstracting infrastructure management entirely, yet their integration into continuous delivery pipelines introduces novel performance, cost, and observability challenges. This paper presents a comprehensive empirical evaluation of serverless-native DevOps workflows, combining platform benchmarking experiments with a multi-organization case study involving six companies that migrated production workloads to serverless architectures. Our benchmarks quantify cold start latency distributions across AWS Lambda, Azure Functions, and Google Cloud Functions under varying memory configurations, runtime environments, and invocation patterns. We find that cold starts impose median latency penalties of 180–740 ms depending on runtime and configuration, with Java and .NET runtimes exhibiting the highest variance. We introduce four cold start mitigation strategies — Provisioned Concurrency, Keep-Warm Scheduling, Lightweight Runtime Selection, and Dependency Minimization — and evaluate their cost-performance trade-offs. Case study findings reveal that serverless adoption fundamentally reshapes team topologies, reducing infrastructure operations burden by 65% while requiring new competencies in cost observability and function granularity design. This work provides the most comprehensive empirical treatment of serverless DevOps integration to date.

Kenji Watanabe, Amara Diallo, Florian Huber, Priscilla Nwosu· Jul 2019· 334 citations