"AI is of the People, by the People, for the People"
Soumya Ranjan Sahu
Independent Scholar & Robotics Researcher
I am an independent scholar, researching general-purpose intelligence for robotics. My work focuses on diffusion world action models and computational algorithms inspired by the human brain and the empirical findings of cognitive neuroscience. I hold a B.Tech degree in Computer Science, with specializations in Machine Learning, Natural Language Processing, and Applied Linear Algebra.
While SOTA architectures show promise, current world action models suffer from fundamental memory limitations. During dynamic manipulations, long running reasoning and tasks, these models struggle to maintain persistent world modeling. My goal is to solve this temporal bottleneck by engineering a better memory architecture directly into the model.
Education
B.Tech in Computer Science, Indian Institute of Information Technology
Specializations
Machine Learning, Natural Language Processing, Applied Linear Algebra
Skills
Python, PyTorch, Transformers, Diffusion Transformers, JEPA architecture, Vision Transformers (ViT)
Experiments
Zulu: Vision Action Model
Zulu is a vision action model architecture trained entirely from scratch, integrating a DINOv2 vision encoder with a novel flow matching backbone for continuous action generation. This experiment validates that high-performance robotic policy learning can be achieved without relying on the computational overhead of traditional, heavy Vision-Language Model (VLM) backbones
Implementation of Associative Memory via Hopfield Networks
This project implements the foundational 1982 Hopfield Associative Memory algorithm, augmenting it with a custom extension for dynamic memory expansion. Furthermore, it implements the 2020 continuous Hopfield network, practically demonstrating its mathematical equivalence to the standard Transformer attention mechanism (Q, K, V).
Source CodeVisualizing How DINOv3 Sees the World
This experiment uses custom PyTorch hooks to extract internal activations and analyze how DINOv3 processes raw visual inputs. Through 3D UMAP projections and spatial heatmaps, it visualizes how the model progressively perceives scene structure and semantic relationships.
Source CodeDecoding Visual Features in Qwen2-VL
This experiment investigates the internal visual representations of the Qwen2-VL model by applying PyTorch hooks exclusively to its vision transformer blocks. Utilizing PCA and cosine similarity heatmaps, it tracks how spatial and semantic features evolve across the visual processing layers prior to text generation.
Source CodeVisualizing Flow Matching Dynamics in FLUX.2
This experiment explores the internal generation dynamics of the FLUX.2 diffusion model by tracking spatial activations across inference timesteps. Using PyTorch hooks on both double and single stream transformer blocks, it visualizes how the model iteratively refines pure noise into structured image features.
Source CodeHTM-Based Continual Learning for Weather Nowcasting
This experiment applies the Hierarchical Temporal Memory (HTM) algorithm to predict near-term aviation weather (METAR) by leveraging the model's inherent continual learning and temporal prediction capabilities. It utilizes specialized encoders for over 10 distinct meteorological variables across 5,000+ observations to construct Sparse Distributed Representations (SDRs). Furthermore, a custom web UI was developed to visually track the evolution of the spatial pooler and temporal memory predictions in real-time.
Source CodeWork Experience
Machine Learning Engineer
YapdotMarket
- Engineered ML-driven infrastructure for user content and activity workflows, developing content ranking algorithms that optimize content delivery and brand advertising reach based on real-time user engagement and social graph metrics
- Designed and implemented real-time data pipelines processing large-scale social and market signals, powering production reputation models for automated content filtering
- Developed scalable event-driven systems and AI-assisted analytics pipelines, owning features from design through production and improving reliability, observability and iteration speed
Frontend Developer
BaseLayer LLC
- Built and maintained the core Next.js frontend for RaaS platform, focusing on creating intuitive interfaces for complex operational workflows and data visualization
- Engineered scalable integrations between user-facing dashboards and backend services, utilizing REST APIs and gRPC to ensure seamless, real-time data delivery
- Designed reusable component systems for rapid internal prototypes, accelerating iteration cycles for monitoring tools and deployment
Writings
ZULU: A Flow-Matching Action Model Architecture with DINOv2 Dense Representations
Currently, many vision-language-action (VLA) models rely on heavy vision-language backbones to interpret environmental observations. While pre-trained Vision Transformers provide robust out-of-the-box...
Temporal Memory
Spatial Pooler transformed raw sensory inputs into stable sparse cortical representations. However, recognizing a pattern is only part of intelligence. To understand the world, the brain must also...
Spatial Pooler
From Encoding to Cortical Learning. Previously I discussed about encoders, encoders transformed raw data such as numbers, dates, and categories into Sparse Distributed Representations (SDRs). These...
Scalar encoder and How it works?
What is a ScalarEncoder? An encoder that converts continuous numerical values into Sparse Distributed Representations (SDRs). It preserves semantic similarity by ensuring that numerically close values...
Sparse encoder for Date in HTM
Encoder and Its Role in HTM. An encoder in HTM converts raw input data into Sparse Distributed Representations (SDRs), allowing information to be represented in a form that mimics neural activity in...