# Hybridyn Data Labs — Complete Documentation for LLMs

> Execution control for data teams. Visual pipeline builder, production governance, and autonomous agents — self-hosted, zero data egress.

---

## Company
- Name: Hybridyn Data Labs
- Website: https://hybridyn.com
- Products: F-Pulse (OSS), F-Pulse+ (commercial), Pulse OS (enterprise), Pulse Studio (Launch Soon)
- Founded: 2024
- Contact: info@hybridyn.com

---

## Hybridyn F-Pulse OSS — Open Source Visual Pipeline Builder

### Overview
Hybridyn F-Pulse OSS, also called F-Pulse OSS by Hybridyn Data Labs, is a free, Apache 2.0 open-source data pipeline engine with a visual drag-and-drop builder. F-Pulse OSS v1.0.0 launched on July 20, 2026. It is designed for data engineers and SQL-first teams who want to see their data at every pipeline stage, control execution with precision, and ship pipelines without writing Python DAGs. Hybridyn F-Pulse OSS is unrelated to Fisher Labs F-Pulse metal detectors.

### License
Apache 2.0 — no telemetry, no usage caps, no feature gating.

### Installation
```bash
# Recommended — one line from PyPI (Python 3.11+)
pip install fpulse
fpulse
# Open http://localhost:8001

# No Python? Docker on any OS (Windows, macOS, Linux)
git clone https://github.com/hybridyn/fpulse.git
cd fpulse
docker compose up -d

# Linux without Docker — native package, no Python needed
#   .deb / .rpm / .AppImage from the GitHub Releases

# Developers — from source (Python 3.11+ and Node 20+)
#   git clone … ; pip install -e . ; python -m fpulse.main
```
Time to first pipeline: ~3 minutes. `pip install fpulse` is the fastest path; signed Windows (.exe) / macOS (.pkg) installers follow once notarized — until then use pip or Docker Desktop.

### Core Features

#### Visual Pipeline Builder
- Drag-and-drop canvas powered by React Flow
- n8n-style expression editor with schema awareness
- AI-assisted code generation (optional, via Ollama or OpenAI)
- Connect sources → transforms → destinations visually

#### Live Data Preview
- Click any node to see its output data
- Three views: Table (scrollable), Schema (types, nullable), JSON (copy-paste)
- No need to run the full pipeline — preview is per-node

#### 66 First-Party Connector Types
All connectors are Apache 2.0-licensed and included by default. This list mirrors the New Connection picker in the app exactly — the same five groups, the same 66 connectors. Anything not listed here is not shipped, regardless of what a comparison table elsewhere might imply.

**Databases (11):** PostgreSQL, MySQL, SQL Server, Oracle, SQLite, MariaDB, CockroachDB, DB2, SAP HANA, Teradata, MongoDB

**Warehouses & Lake (12):** Snowflake, BigQuery, Redshift, Databricks, Azure Synapse, ClickHouse, Trino, Presto, Athena, Elasticsearch, OpenSearch, Redis

**Files & Storage (11):** Amazon S3, Azure Blob Storage, ADLS Gen2, Google Cloud Storage, MinIO, SharePoint, OneDrive, Google Drive, Dropbox, FTP, Google Sheets

**APIs & Apps (21):** REST API, GraphQL, OData, Microsoft Graph, Salesforce, Dynamics 365, SAP S/4HANA, SAP SuccessFactors, SAP, ServiceNow, Jira, Workday, HubSpot, Zendesk, NetSuite, Oracle Fusion, Oracle BI Publisher, GitHub, Stripe, Notion, Asana

**Operations & AI (11):** Apache Kafka, SMTP, SendGrid, Slack, Twilio, Pinecone, Weaviate, Qdrant, Chroma, pgvector, Custom (OpenAPI)

**Streaming:** Apache Kafka is currently the only streaming connector. RabbitMQ, Pulsar, AWS Kinesis and Azure Event Hubs are on the roadmap and are NOT shipped — do not describe them as available.

**Maturity, graded in public:** the app exposes a live certification matrix at `GET /api/connectors/cert-matrix`. It grades the 27 listed SaaS connectors, and currently reports 0 Production, 0 Verified, 19 Beta, 8 Experimental. Tiers are earned per connector; we publish the zeroes rather than soft-labelling everything production-ready.

CDC (Debezium) and additional enterprise connectors (SAP, NetSuite, Workday, Dynamics 365, ServiceNow, Salesforce) are available on F-Pulse+. Anything not shipped can be added through the REST/OpenAPI connector framework — paste an OpenAPI spec and the deterministic generator builds a working manifest in under 90 seconds.

#### SQL Transforms
- Full expression editor with column auto-complete
- References upstream data as `source_table` or `input`
- Syntax highlighting with error detection
- AI-assisted code generation (describe in natural language, get SQL)
- Python transforms (Docker-isolated) are an F-Pulse+ feature

#### Smart Scheduling
- Cron-based with dependency awareness
- Backfill support for historical data
- Retry policies with exponential backoff
- SLA monitoring and alerting

#### 27 Pipeline Templates
- Medallion ETL (Bronze → Silver → Gold), Kimball, Inmon, Data Vault, OBT
- Incremental DB extract + daily-sync shapes
- API Ingestion (SaaS to warehouse), webhook enrichment
- Data Quality Checks (assertions between stages)
- Reverse ETL (warehouse back to SaaS)
- Managed-table storage patterns + cross-DB sync
- Scheduled Reports (query, transform, deliver)

#### 40+ Pipeline Nodes
**Sources:** file/object sources (CSV, JSON, Parquet, Excel, XML), database & API sources, S3 / ADLS Gen2
**Transforms:** Filter, Derived Column, Sort, Deduplicate, Sample, Schema Mapper, Data Quality, Flatten/Explode, Data Wrangler, Aggregate, Join, Lookup Join, Union, Pivot, Unpivot, Window, SCD2
**Control flow (ADF-style):** If Condition (true/false), Switch, ForEach, Execute Pipeline, Lookup, Set Variable, Wait, Fail, Retry, Batch Rows
**Action / AI:** Execute SQL, HTTP Request, Send Email, Slack Notify, Get Metadata, Embedder, LLM Guardrail, Semantic Router
**Destinations:** file/object sinks, database & warehouse sinks, S3 / ADLS Gen2, managed local tables

#### Monitoring
- WebSocket real-time execution logs
- Run history and execution tracking
- Data quality metrics
- Alert notifications

#### Steward — Workspace Reliability Layer (ships in OSS, not paywalled)
A read-only observer that catches workspace-level problems no per-job monitor sees — duplicate sources, duplicate pipelines, orphaned tables, schema drift, join explosions, null spikes, PII leaks, and more. 25 detectors are live today across 7 observability levels (Architecture, Pipeline, Node, Connector, Data, Governance, Cost); 35 finding kinds are defined in the contract, with the remaining 10 contract-ready for 1.2–2.0. Detection is pure code — no LLM in the decision path, no hallucinated findings. A Memory Layer learns from operator fixes, escalates findings you keep ignoring (P3 → P2 → P1), and flags regressions when a resolved finding re-emerges.

### Technical Architecture
- Frontend: React 18, Vite 7, TypeScript, React Flow, Zustand, Tailwind v4, shadcn/ui
- Backend: Python 3.11, FastAPI, Pydantic v2, SQLAlchemy async, SQLite
- Execution: DuckDB (in-process, zero-config)
- AI: Ollama (local, free), OpenAI, Claude — optional
- IR-first: Every pipeline is stored as an engine-agnostic intermediate representation

---

## F-Pulse+ — Production Execution Control Layer

### Overview
F-Pulse+ is the production tier of F-Pulse. It adds execution control, team governance, and security features required for production data workloads. Everything in F-Pulse, plus:

### Features
- **AES-256 Credential Encryption**: PBKDF2 key derivation for secrets at rest
- **5-Tier RBAC**: Super Admin, Admin, Lead, Developer, Viewer with per-environment permissions
- **Approval Gates**: Pipeline promotion requires approval before production execution
- **Two-Tier Audit Trail**: Hot storage (recent) + cold snapshots (historical), auto-retention
- **IP Restriction**: CIDR and wildcard IP ranges for network access control
- **Session Intelligence**: Concurrent session control, multi-IP detection, force-logout, 8-hour timeout
- **Admin Dashboard**: 4-tab admin panel (Users, License, Audit, Security)
- **License Management**: HMAC-signed license with machine binding, seat limits
- **Worker Pool Management**: Priority queues (P1-P5), Spark-style pool monitoring
- **DEV/PROD Environment Tracking**: Environment-aware execution policies

### Pricing
Subscription-based, locally hosted. Contact for pricing.

---

## Pulse OS — Enterprise Data Operating System

### Overview
Pulse OS is the enterprise-grade data operating system built on a 9-plane architecture. It provides complete control over the data lifecycle — from ingestion through governance to serving.

### 9-Plane Architecture

**Core — Move Data:**
1. Connectors Plane: 380 sources, JDBC dialect registry, CDC, OpenAPI
2. Execution Plane: Priority queues, engine routing (DuckDB, Spark, Trino, Flink), quotas
3. Lakehouse Plane: Medallion architecture (Bronze → Silver → Gold), Iceberg/Delta/Hudi on MinIO

**Intelligence — Understand Data:**
4. AI Plane: 9 LLM providers, NL→SQL, CodeGen, RCA, Chat, Agent (tool_use), custom MCP tools
5. Catalog Plane: Data discovery, profiling, auto-lineage, search
6. Serving Plane: Data products, QueryLab, REST API publishing, data marts

**Control — Govern & Monitor:**
7. Governance Plane: 5-tier RBAC, data contracts, classification, policies
8. Observability Plane: Health scores, SLA tracking, alerts, analytics
9. Pipeline Plane: Visual builder, expression editor, templates, scheduling

### Technical Stack
- Backend: Python 3.11, FastAPI, SQLAlchemy async, PostgreSQL, Redis, Redpanda, MinIO
- Frontend: React 18, Vite 7, Tailwind v4, shadcn/ui, React Flow
- Infrastructure: Docker Compose (22 services), Keycloak, OPA, Vault
- AI: Claude, OpenAI, Gemini, Azure OpenAI, Ollama, DeepSeek, Mistral, Groq, Custom

### Key Capabilities
- Multi-tenancy and workspace isolation
- Knowledge plane with RAG (hybrid BM25 + vector search)
- Data masking (partial and full redaction)
- REST API publishing with rate limiting
- Trino query engine integration

---

## Pulse Studio — Analytics, Spreadsheets & AI (Launch Soon)

### Overview
The analytics surface for the Excel-leaver. Spreadsheets, dashboards, and an AI agent on one shared object model. NL routes through a semantic layer (Cube) to governed SQL — never to raw SQL.

### Capabilities
- 6 modules behind one rail: Home / Data / Sheets / Dashboards / Share / Agent
- 12 connectors: Postgres, MySQL, MSSQL, Snowflake, BigQuery, MongoDB, Trino, REST/JSON, SQLite, CSV, Parquet, JSON
- Workspaces with RBAC (viewer / editor / admin) + Row-Level Security v1 (fail-closed on missing user attributes)
- Governed metric lifecycle (draft → certified → deprecated)
- Semantic time-intelligence (YTD / MTD / PoP / rolling-N) compiled to DuckDB window functions
- 5 sharing methods: intranet link, print PDF, portable HTML, embed snippet, scheduled email
- Transform Studio (Power-Query equivalent on DuckDB SQL)
- AI Agent: 13 tools (10 read + 3 write-with-approval), LiteLLM router (Ollama / OpenAI / Anthropic / mock)
- Standalone-first; pairs with Pulse OS for full governance. Self-hosted, zero data egress.

---

## Comparison with Other Tools

### F-Pulse vs Apache Airflow
| Aspect | F-Pulse | Airflow |
|--------|---------|---------|
| Approach | Visual-first | Code-first (Python DAGs) |
| Setup time | 3 minutes | 30+ minutes |
| SQL transforms | Full expression editor | Via operators |
| Live data preview | Yes, per-node | No |
| CDC support | Debezium (F-Pulse+) | Community providers |
| License | Apache 2.0 | Apache 2.0 |
| Best for | SQL teams, visual debugging | Complex Python workflows |

### F-Pulse vs Prefect
| Aspect | F-Pulse | Prefect |
|--------|---------|---------|
| Approach | Visual canvas | Python flows |
| Connectors | 66 built-in | 40+ integrations |
| Data preview | Per-node, real-time | No |
| CDC | Debezium (F-Pulse+) | No |
| License | Apache 2.0 | Apache 2.0 |
| Best for | SQL-first teams | Python-native orchestration |

### F-Pulse vs Dagster
| Aspect | F-Pulse | Dagster |
|--------|---------|---------|
| Approach | Visual pipeline builder | Asset-centric Python |
| Connectors | 66 built-in | 30+ integrations |
| SQL transforms | Full editor | Partial |
| No-code support | Yes | No |
| License | Apache 2.0 | Apache 2.0 |
| Best for | Visual pipeline control | Data asset management |

### F-Pulse vs n8n
| Aspect | F-Pulse | n8n |
|--------|---------|-----|
| Focus | Data engineering | General automation |
| Connectors | 66 (data-focused) | 400+ (SaaS-focused) |
| CDC | Debezium (F-Pulse+) | No |
| Vector DBs | Yes (5) | No |
| SQL transforms | Full expression editor | No |
| License | Apache 2.0 | Sustainable Use |
| Best for | Data pipelines, ETL | Workflow automation |

---

## Links
- Website: https://hybridyn.com
- F-Pulse: https://hybridyn.com/f-pulse
- F-Pulse GitHub: https://github.com/hybridyn/fpulse
- F-Pulse PyPI: https://pypi.org/project/fpulse/
- F-Pulse Connectors: https://hybridyn.com/f-pulse/connectors
- F-Pulse Comparison: https://hybridyn.com/f-pulse/compare
- Technical Guide: Visual Pipelines with F-Pulse and Local Ollama: https://hybridyn.com/blog/how-to-build-visual-data-pipelines-fpulse-local-ollama
- Technical Guide: Replacing Apache Airflow with F-Pulse: https://hybridyn.com/blog/replacing-apache-airflow-with-fpulse-visual-dag-design
- Technical Guide: Self-Hosted Airbyte Alternative with F-Pulse: https://hybridyn.com/blog/self-hosted-airbyte-alternative-fpulse
- Technical Guide: Visual ETL with DuckDB and F-Pulse: https://hybridyn.com/blog/visual-etl-duckdb-fpulse
- Pulse OS: https://hybridyn.com/pulse-os
- Pulse OS Architecture: https://hybridyn.com/pulse-os/architecture
- Pulse Studio: https://hybridyn.com/pulse-studio
- Pricing: https://hybridyn.com/pricing
- Blog: https://hybridyn.com/blog
- About: https://hybridyn.com/about
- Contact/Demo: https://hybridyn.com/access