Skip to content
ecbyts

← All news

Engineering

How ecbyts builds the proprietary ECBT language model

About 5 min read

  • Guide

Built in PyTorch with a deterministic dual track, the compact ECBT model separates language reasoning, exact backend math and gap control without inventing field numbers.

How ecbyts builds the proprietary ECBT language model: cover with artificial intelligence, data, rules and technical analysis icons.

General-purpose language models write fluent prose, yet break down on three essential requirements in environmental engineering: they guess intermediate numbers when asked to compute loads or dilutions, overlook missing monitoring campaigns and require uploading confidential project files to external servers. Since July 2026, ecbyts has developed ECBT in-house in pure PyTorch specifically around digital twins, hydrogeology and operational governance.

Why generic language models fail environmental engineering

When engineers inspect multi-year monitoring tables mixing concentrations in mg/L and flow rates in m3/d across dozens of wells, off-the-shelf assistants drift across unit bases and fill observational gaps by next-token probability. Without an unbroken link back to each source observation, generated figures lose technical defensibility.

Compute footprint and provenance pose an equal challenge. Frontier open weights demand multi-GPU cloud clusters, while standard engineering workstations carry strict memory ceilings. Relying on opaque third-party weights also removes control over training dataset licenses, personal data filtering and contamination checks against held-out benchmarks.

How ecbyts solves it

How ecbyts builds the proprietary ECBT language model: PyTorch core and deterministic dual track separating reasoning from backend calculation.

Across more than 2,100 repository commits, ECBT researches hybrid Transformer building blocks pairing compressed latent attention (GatedMLA) and recurrent linear delta attention (KDA) with Mixture-of-Experts (MoE) routing, YaRN positional scaling and Multi-Token Prediction (MTP). For product deployment efficiency, engineering prioritized compact architectures including a 394-million-parameter `student` track and a 206.2-million-parameter dense `a10-Fast` profile, alongside a 274K-parameter `tiny` rig for continuous regression gates.

At the core of ECBT sits a Dual-Track architecture separating raw neural weights (Track A) from a deterministic system layer (Track B). Neural layers parse intent, ingest structured `[Fonte N]` evidence blocks, surface declared missing measurements and propose typed JSON tool payloads. Deterministic backend software executes every unit conversion, daily mass load calculation (`C x Q / 1000`) and regulatory threshold comparison. The language model never acts as an arithmetic source of truth.

Data preparation obeys strict provenance rules. Direct tokenizer measurements showed naive characters-divided-by-4 rules undercount real token budgets by up to 23.39% on Portuguese technical docs (measured at 3.674 chars/token overall and 3.064 on Portuguese technical text), motivating individual-digit tokenization for numerical stability. Training mixes undergo subset-by-subset license screening, disjoint isolation of 21,196 held-out items (`9.77%`), and automatic personal data redaction tokens (`[CPF_1]`, `[EMAIL_1]`, `[TEL_1]`) before any request leaves internal boundaries.

In practice

On November 16, 2026, after fixing a two-step shifted-label bug and removing repetitive fine-tuning templates, the dense 206.16M-parameter from-scratch track completed a 4,850-step pre-training probe across 1.271 billion tokens. Measured bits-per-byte (BPB) dropped steadily from 2.8395 at start to 1.7229 at 500M, 1.6142 at 1B and 1.4871 at 1.271B tokens, confirming real language modeling progress, while the clean 3-shot gate scored 0.25% (1/400) due to sequence repetition loops typical below 3 to 4 billion tokens.

Guided by those open diagnostics, autonomous bilingual curation (Portuguese and English) is scaling the educational corpus from 1.8 billion toward 4.5 billion tokens (adding 2.7 billion vetted tokens under SHA-256 artifact manifests and fail-closed `resource_guard` local memory caps) ahead of full pre-training and a diverse 6,000-example fine-tuning stage.

Controlled evaluations on the `v4d` 393.6M-parameter student checkpoint quantify the Dual-Track impact. Across 400 held-out items, raw weights climbed from 9.0% baseline accuracy to 21.75% (87/400), while full Track B system integration reached 59.50% (238/400). On a disjoint 96-item fresh split across eight technical categories, raw weights (Track A) scored 22.92% (explicitly reported below a 35% standalone target), whereas Track B raised accuracy to 56.25% (54/96), achieving 12/12 on gap-calibrated evidence grounding, 11/12 on agentic tool syntax and 12/12 on technical translation.

Security and hardware contracts enforce operational discipline: `ECBT_MODE=public` refuses startup without credentials, stores keys only as SHA-256 digests, rejects wildcard CORS and enforces safe tensor deserialization (`weights_only=True`). Under a declared single-GPU `bf16` server contract, the serving runtime measured 129.0 tokens/s decode throughput.

What this feature is not

  • The dense 206.2M from-scratch track remains in early scaling: although its 1.271B-token probe lowered BPB to 1.4871, clean 3-shot accuracy stood at 0.25% (short of the 28% and BPB 1.14 readiness gate) due to repetition loops, so ECBT is not published as an open sequential chat before the full 4.5B-token run and fine-tuning complete.
  • Standalone weights (22.92% on the fresh split) are never shipped unsupervised for client calculations: measured reliability depends on the deterministic tool and grounding layer.
  • Model outputs never replace physical sampling, accredited laboratory reports, process simulators or sign-off by a licensed engineer under the ECBYTS Seal.
  • Measured inference rates (129.0 tokens/s in dedicated GPU tests) apply strictly inside their declared hardware, precision and concurrency contract, not consumer 4 GB cards.

How to practice

Read more about how deterministic models ground artificial intelligence in the articles on deterministic digital twins and AI and the ECBYTS Seal for AI-assisted reports. Explore the live workspace on ecbyts or reach engineering via the contact page.

← All news